I personally used this guide to overclock my i5 2500k to 4.5 ghz (although I don't have the stock cooler). First you should run Prime95 and RealTemp to see what are your temperatures when cpu is on full load. Normally if you are under 60 you should easily overclock to 4ghz. I was actually able to overclock to 4 ghz by lowering the voltage from what was previously used with the "auto" setting.

All chips are different. Just set a 40x multiplier on all cores and try leaving it on auto voltage. If it's more than 1.3v on auto, I would manually back down and try around 1.2v or even lower depending on how well your chip overclocks. Run prime95 to test for stability.

At this point, it's going to come down to what temps you are comfortable running while at load. Everyone has their set point temp they don't want to go above. The CPU will throttle down at 95C in order to prevent immediate damage to the CPU. This isn't to say slightly lower than 95C is ok to run at for extended periods of time. The problem is that the higher temp your running your cpu, the greater the chance of degradation. Personally, I won't run a cpu at full load higher than 75C unless I have enough cash to go out and replace it at will. With this in mind, if I were you I wouldn't bother overclocking that until you get yourself a $50.00 decent heatsink/fan. Just my opinion.