Nope, no reason at all. In fact it's better for the moulds to demould earlier, as a degree of flexibility of the part means less stress on the silicone. Through experimenting you'll find the optimal time after which to demould and it will depend on the part and mould design. Too early and you'll warp your part too much as you demould it (which isn't the end of the world, as you will be able to straighten the part again post curing with a heatgun, but it's better to avoid this).