Open MPI does specifically test each of the C, C++, F77, and F90
compilers for the -pthread flag (and others). When mixing gcc/g++ and
pgf77/pgf90, OMPI's configure script correctly determines that gcc/g++
support -pthread, but pgf77/pgf90 do not.

Note the addition of -pthread, which then causes the problem. I
*suspect* that this is because the f90 library is linking against
libmpi, libopen-rte, and libopen-pal (OMPI internal libraries) that
were built with -pthread (i.e., Libtool picks up these flags
automatically). This should probably be reported to the Libtool
developers, but I'm not 100% sure they can fix it -- I believe that
they assume that the linker flags used for one language can be used in
any language (compiler/linker).

I can think of two workarounds for you (one significantly less icky
than the other):

1. Use pgcc, pgCC, pgf77, and pgf90 to build Open MPI. If you have no
C++ MPI code, the resulting Open MPI build *should* be compatible with
your C + Fortran code.

2. Instead of using the "real" pgf77/pgf90, put pgf77/pgf90 scripts
early in your PATH that simply strip out -pthread from the argv and
then invoke the real/underlying pgf77/pgf90 compilers. This is pretty
icky, but it should work...