These are all warnings caused by mixing enums and ints as though they
were interchangeable. This concerns me, because, one can mistakenly
assume the only possible values of an enum data type are the range of
valid enum constants. That can make debugging such errors very
difficult. I do not know if you are aware of these.

I would think mca_btl_openib_modex_message_t.transport_type should be
a compatible enum as well. At least there should be some code
somewhere with asserts or validity checks in, say, a switch block when
DEBUG=1, and a cast otherwise.

The warning is really an error, since neither the right hand side
"IBV_QP_ALT_PATH|IBV_QP_PATH_MIG_STATE" nor "0" is one of the enum
ibv_qp_attr_mask values, and is therefore not a valid member of that
data type. I'm not really sure there is a better way, since C does
not guarantee the order of bitfields. Anyway, since C permits enums
to be used wherever ints can be used, the right hand side of

> *mask = IBV_QP_ALT_PATH|IBV_QP_PATH_MIG_STATE;

is equivalent to

> *mask = (int) IBV_QP_ALT_PATH | (int) IBV_QP_PATH_MIG_STATE;

, which results in an int. (And, of course, = 0 is an int.)

The simplest fix would be to cast the result into an enum
ibv_qp_attr_mask, with comments added that enum ibv_qp_attr_mask *mask
is really the union of all the bitfields in enum ibv_qp_attr_mask, and
that the value of *mask may not be a valid enum ibv_qp_attr_mask.