So considering the results are identical, which configuration would be better (if any?) 2 things that I see different is the cost and the layout of the components. But if the cost and layout are not an issue which configuration would give better sound? I think if you have a lot of cells with smaller value components could create oscillations. If there is no issue regarding oscillations how about a configuration with a lot of cells (say 10 cells with 5uF / 0.2H). Would that work too?

SE amp or PP, it does make a difference both in how critical supply transient response is and the amount of ripple you can tolerate on its output. In general a well damped step response is what you are shooting for. SE amps have very poor PSRR unless parafeed so ripple level becomes important.

Try decreasing C2, even if it'll be lower than C1 (this I am not sure and you have to try). If it doesn't work, you'll have to increase C3. Basically the smaller the middle cap and the larger the end cap the smoother will the curve be.