Whats the gate - source voltage on the large mosfet?To turn it on hard so that its dissipation is low, you will need close to 20V.5 V wont turn it on hard , so its dissipation will be high, hence the heat build up.

Yes it is quite possible to 'smoke' a power MOSFET even when operating at less then it's max rated current. To try and figure out what you may have wrong a few questions:

Are you PWM switching the lamp to simulate a dimming ability?

Can you draw up your complete wiring of the small mosfet driving the large mosfet and the lamp? I'm betting that you do not have a active current sink and current source driving the gate of the larger mosfet and because the larger mosfets have significant gate capacitance that must be charged and discharged as fast as possible or there will be higher heat dissipation during the switching transitions that could be your source of problem.

The IRF3205 is not logic level so you can't expect it to switch with 5V - it's spec'd for a minimum of 10V gate voltage.

The only important specs of a switching MOSFET are voltage and the Rds(on) value and the Vgs value that goes with it. In most cases I've seen the maximum current is simply the maximum power rating with infinite heatsink (recast as a current value). Unless you are providing water-cooling you won't be running at anything like that current!

A 12V 50W halogen will take about 4A steady state, so for less than 0.25W dissipation (no heatsink required), you'd choose Rds(on) of 15milliohms or less. Here the IRF3205 is fine (so long as you drive it with 10V or more).

So clearly you are either failing to provide 10V drive, or your circuit is wrong (using source-follower rather than common-source configuration perhaps?)

How did you work out that the bulb has a resistance of 3ohm when fully lit?

What is that 15x15x8mohm calculation called, so I can look it up and study it a bit.

I am aware you can also calculate how much temperaure a heatsink will suck off the mosfet. But what size do I get 1" x 1" size?Ahh, just had a look at it in a catalog and it says a degreeCelcius/Wexample there is one here that is for a to-220, that is 19degree celcius/W (watt I assume)

How does this work then? It will suck away 19 degrees for every watt of power?Then if I look at the much larger heatsinks they are rated at like 1.4degrees celcius/W, I know they will take the heat away better than the smaller ones but they have a lower temp/watt value???

Okay I have made up a diagram, and have noticed a wrong hook up. You will see a red line where it is wrong.

Yes it is connected to a PWM output on the arduino.

I have a smaller mosfet to turn the bigger one one, like a darlington pair I guess.

See the schmatic.

Yes, that is a mess and the red wire is confusing, it's wrong because it's there or it's wrong because it wasn't there and you added it? Either way that is not a good gate driving circuit because it's relying on a passive pull down resistor to turn the large mosfet off, which means it has to allow for a time constant of 10k ohms X the mosfet gate capacitance to fully turn off, which means the mosfet is dissipating significant heat while in that slow transition time from on to off.

A far better solution would be to just use a 'logic level' power mosfet to begin with and eliminate the need for a 'driver' mosfet. Here is an example that would work well. https://www.sparkfun.com/products/10213

OK that driving circuit is plain wrong. The first device (for which you've used an NPN symbol, not an n-channel FET) is in source-follower mode - this is no good at all, it won't pull up the second devices gate more than a volt or two.

The first device must be a common-source stage, with a drain load resistor to the +12V rail (something like 1k perhaps). This will level convert from 5V to 12V (and invert the logic). This can then drive the second devices gate at suitable levels to actually turn it on.