Cream butter and sugar in a large bowl. Add eggs and sweet potatoes, and blend in well. Combine the flour, baking soda, salt, cinnamon and nutmeg in a medium bowl. Add to the butter mixture, alternating with the milk, until combined. Fold in nuts, if desired. Fill prepared muffin tins 3/4 full and bake for 20 to 25 minutes for regular muffins, 15 minutes for mini muffins.