Tough call, they're both great lenses with unique attributes. The 300 f/4is is one of Canon's classic "sleeper" lenses. I regret selling mine when I got the 300 f/2.8is. It's just so light and compact in the bag, justifying a permanent spot in the bag. Most copies are pin sharp wide open. Mine responded well to the x1.4 extender, giving you a 420 f/5.6.

For what you have described, the 300 f/4is may be the best bet. It's a stop faster than the 70-300 f/4-5.6 at the long end for a start, plus you already have the 70-200 f/4.

If you can locate a pre-owned 300 f/4is you probably wouldn't lose a penny on it if you decided it was not right for you.

I really enjoyed using the 300mm f4L. It has quick accurate focusing and is very sharp. The 1.4xII (not III) is a perfect companion with it. I purchased both items used for less than the cost of the 300 new.

Go for the prime, you will not be disappointed

+1 on the 300 F4 L IS. IQ ia significantly better than the 100-400 (I had a good example). The 300 takes the 1.4 extender very well. It can also (in good light) produce passable images with the Canon 2 x Mk3 extender but you may have to resort to live view focus.

+1 also... Used this lens for many years until recently picking up the 2.8. Razor sharp and it really does shine with the 1.4x ext II. Never came away with quality images using the 2x ext III though. Way sharper than the zoom your looking at. My only complaint with the prime was it's IS was a bit jittery with quick targets ( and yes, I was using the correct setting ).

Be careful of sample variation. My 300 F4 L IS needed to go in four times before it was right (bought new from Adorama).

When that lens is "right", it's the best bang for buck wildlife lens that Canon makes. I know many wildlife photogs who sold their 300 2.8's for the F4 version, claiming relatively similar results, but far less weight and increased portability. The 600 F4 and 300 F4 is a popular pairing.

This was taken with the 300 bare, no processing, pretty much the straight RAW file at ISO 1600 handheld.