I think the two big deterants for regular citizens wearing body armor is the cost and comfort issues. If you, as an off-duty LEO have been accustomed to wearing it and have access to one for duty then I wouldn't think wearing one when you wanted that extra layer of peace of mind is out of touch at all.