You can take them out at any time. They do not interfere with breastfeeding (well obviously you have to take them out first, but you can still produce milk), and they won't stretch your nipples out, breastfeeding does that