Slow response

HI,
I am without resolve on this one, out of the blue, one of my servers thats been running for weeks now, suddenly become almost impossible to log into. the login process from the consile / ssh and even all ports 110 ans 25 seem tot timeout to a couple minutes!!
My initial thoughts was that his server run out of some resource or another. however I can not detect the resource nor the process in error.

By using top i can indicate an running system - althow slow it seem to function

It take 30-40 minutes ands some services like MYSQL fail to stop, eventually it seem to reboot.
If i stop postfix, no change.What can I test for, how can I seek the problem.

I had this issue with an smaller server (mandriva+mailbox), replaced it eventually with (ubuntu 7.04+maildir), but I cant keep on re-building and replacing servers, instead of repairing some missbehaving part.

1) Do not write in big red letters. This might lead the one or other user to not answer to your post. We are all able to read the normal small black letters

2) The problem is that your server receives to much mail for the currently installed spam scanning and antivirus solution with clamscan and spamassassin. Check your mail log if you get higher email traffic, e.g. a spam.

and let it do its thing - analysing the top on an terminal and looking at the logfiles to see what processes are closing off, obvious that the once the process causing the issue shuts down the re-boot will speed-up.
All the services stopped, took loooong time from 15:13 to last entry in syslog at 16:01, almost an hour (48 minutes!) the last standing processes was called from ISPConfig clamscan and authdaemond All the time the HDD seem to run non-stop!

1) Do not write in big red letters. This might lead the one or other user to not answer to your post. We are all able to read the normal small black letters

2) The problem is that your server receives to much mail for the currently installed spam scanning and antivirus solution with clamscan and spamassassin. Check your mail log if you get higher email traffic, e.g. a spam.

Click to expand...

Oops big letters came as I did not preview, ancase fixed.

I have received the same mail 3 x during the day, normally indicating pop3 disconnectin and huge mail on bad lines - however this case i have <1ms latency to server on same LAN segment. Thus I was thinking an re-lapse in the bug of clam were it re-serviced already scanned mails over and over again!

This last scenario seem more likely as It takes AGES to work through the mail Q,

Yes it could be the case - what is the solution? this has not happened before on this site as I can indicate with Stats this is an Small site! Normal last week this time's figures.

I left the server to work off the load, as another server of mine seem to have had the same issue was re-booted and that sorted it. This is for sure and spam attack of some sort! it kills clamscan and spamassassin, eventually mysql process gets killed as the server run out of memory! and all get grinded to an halt!
I still have no idea how to get rid of this, were to find the "queue to scan as i stop both postfix and ispconfig_server, clamscan keep on spawning scans. Were is that backlog, how do i remove the heaped-up mail to be scaned! It also seem to scan the same mail over and over, as i got some mail 4-5x already. DARn!

clamd is much faster then the current clamscan solution. Sinvce the clamav developers introduced clamav 0.90.x, the antivirus scanning is producing a much higher load. the problem seems to be known according to the clamav mailing lists and the clamav developers recommend to use clamd. There is also some patch which I hope will be integrated in some of the next clamav releases.

It seems that clamscan could never eat the chunks of mail sent to it, the server become 100% unresponsive and login took hours. clientsatisfaction became Zero, while you see the faxes arriving from customers cancelling the service and shouting "nOOb" all over.
I Backed up the old server after I managed to quickly stop services postfix and ispconfig_server as well as killed all processes to do with clamscan (It dint go away after even 30 minutes as predicted by Till)
It took me exactly 3H15 minutes from the moment I rebooted on the CD to re-install the server till up and running with "Perfect ubuntu 7.04", ISPConfig and Backup restored / user passwd/shadow/group/gshadow files fixed.
However only to be confronted with Exactly the same problem! At that stage Till's feedback arrived and I was confused, he told me to edit the scritpt /home/admispconfig/ispconfig/tools/clamav/bin/clamassassin and fix it to work with clamd/clamdscan. Well it took me to again untar the ISPConfig installation files and search through install_ispconfig directory and found the README for clamassassin, in there was an nice indication of what to do.
The lessons learned here is that it does not always help to use the "Microsoft Fix" i.e. - Reload, but rather seek the real issue, then start digging/googling.

Have you tested your solution with the eicar virus test pattern and does it really filter viruses? You need a running clamd daemon for this and I do not see where you configuare and start clamd automatically as ISPConfig does not ahs a clamd daemon.

I would like to ask ISPConfig to perhaps consider using clamd instead of clamscan by default. I only relised now that I had to upgrade hardware on 2 other installs before this that would have been unnessesary if we switched!
Maybe add an option to switch the two methods.

Yes sir, i tried the test pattern and the mail got filtered out. (ie. its working properly)

Also, clamd daemon is running. Do i need to add that to chkconfig to make sure that clamd daemon is running when system starts?

I am also monitoring my running processes via "top". I only see clamd coming active there are mail activities.

Click to expand...

Yes this is were the issue comes in. Currently you have 2 installations of clamd, one installev via ISPC / clamassasin part and one using yum, Also 2 x freshclam. each updating its database residing in diferent places.
Now, that being the case the versions might be diffrent! also make sure the you inspect the headers of an mail comming through to see the scan successfull! I have this in my header. The No mean there is no virus found!

I was wondering if i could copy /home/admispconfig/ispconfig/tools/clamav/sbin/clamd
to
/etc/init.d
then add it to chkconfig to get it to run when system start?

Click to expand...

This will not work because in /etc/init.d are the start scripts for the daemins and not the daemin binaries itself. If you want to run the clamd daemon /home/admispconfig/ispconfig/tools/clamav/sbin/clamd, you will have to craete a start script for it in /etc/init.d.

This will not work because in /etc/init.d are the start scripts for the daemins and not the daemin binaries itself. If you want to run the clamd daemon /home/admispconfig/ispconfig/tools/clamav/sbin/clamd, you will have to craete a start script for it in /etc/init.d.

Click to expand...

Ah i c.

Thanks. I second Morons's suggestion of havig CLAMD being used as default in ISPCONFIG.