Hopefully there would be an easier way to do a simple thing like this but the problem I cant figure out is
is if you DO NOT want to overwrite the captcha say calling it something like: My_Form_Decorator_CustomCaptchaStyle instead of My_Form_Decorator_Captcha

Comments

I think that the reason for this is that by default Zend_Form_Captcha will add Zend_Form_Decorator_Captcha and so you are basically rendering it twice because both Zend_Form_Decorator_Captcha and My_Form_Decorator_Captcha are being called.

This is not actually a bug so I will mark it as closed, you might have more success querying the mailing list if you are still having issues (form decorators can be a real pain sometimes).