It isn't uncommon to take a break from poly relationships when a new baby is born. You two will be very busy and exhausted dealing with a newborn, so I don't think it's unreasonable to ask him to shift his focus on your marriage and parenting for a while.