Bostjan,
Try a program like MTR or for windows
http://winmtr.sourceforge.net
So you can see which hop is causing your problems.
Check also cables and try to ping from the M0n0wall webgui. Look also
for In/out errors and Collisions on the Status: Interfaces page. It
could be a duplex problem.
@ Chris
Happy birthday better late than not at all ;-)
Maarten