Your defense loses points because they give up the extra point. But the 6 points should not count against the total points your defense gives up. Whether your kicker loses points is based on your league's scoring.