Go with your gut. While it is true they are only little once, it is also true that certain opportunities may only present themselves once. The only right answer is what works best for your family. I swore up and down that I was going to be a stay at home mom, but it wasn't unitl I was in that situation that I realized how much I needed to work and what an enormous benefit it was to my family for me to be happy and confident. For me it took a part time schedule for 1.5 years and then a step up into full time work. For you it may be a complete stop or a step down or no change at all. Don't beat yourself up over the decision. There will be missed opportunities and amazing moments no matter what you decide to do. Either way you will make the right decision.