how to analyze a DOS attack?

I think some script kiddie or similar is having fun targeting my server. happened about 3 times in the last 3 weeks. server would come to a stand still and all I can still see is that all 4GB of RAM is begin used and about 5GB of swapping done. countless apache2 threads and php-cgi processes. Munin show a huge spike in traffic.
everything is becoming so slow that only a reboot can help.

now how would I analyze my log files to see which site was being targeted and which IP or IPs the attack came from?

can one use some iptables rules to block i.e. incoming packets from any IPs that are asking for a site too often, within certain limits?