Using GOD Damn it, OH GOD, and Jesus Christ....Oh my!!

Do you or should you use such phrases and words despite being Atheist? I'm not sure, but I am very guilty of using them anyways, mainly cause of the culture I grew up with in the US. A smidgen of me enjoys knowing that it is meaningless to me but pokes at the offending religion that is Christianity. I really should be a better person and just drop them from my vocabulary completely, yet I have the thought that it would just look like I am justifying or giving power to that religion over me. Your thoughts?

Tangential to this discussion is a usage—common in the South in my experience, but nowhere else—to denigrate another person's foolish opinion or belief without reflecting on the person. For example you might hear something like the following:

She believes—God bless her—that her husband is coming back to her.

He thinks—God bless him—that his business is succeeding.

Is anyone else familiar with this usage? I found it charming and amusing. (God bless me.)