We installed a linux cluster recently. The OS is Ubuntu 8.04. The
network is infiniband. We run a simple MPI program to compute the
value of pi. The program follows three stages: MPI_Bcast, computation
and MPI_Reduce. We measure the elapsed time on computation and
communication, respectively. We also repeat the entire process in a
loop terminated by user input. We run the program interactively using
mpirun. We found an weird problem.

When MPI_Bcast and MPI_Reduce are called for the first time, they are
very slow. But after that, they run at normal and stable speed. Is
there anybody out there who have encountered such problem? If you need
any other information, please let me know and I'll provide. Thanks in
advance.