How to send C++ STL objects in MPI?

The problem that Brock is asking about is that STL objects tend to be variable size and type. The whole point of the STL is to create flexible, easy-to-use “containers” of arbitrary types. For example, STL lists allow you to create an arbitrary length list of a given type.

To cite a concrete example, let’s say that my application has an STL vector object named my_vector that contains a bunch of integers. What parameters do I pass to MPI_SEND to send this beast?

The short answer is: use boost.mpi. boost.mpi is a subsection of the Boost software package that supports several nice C++ language features with MPI, the easiest to cite of which is the ability to send and receive STL objects.

Do not fear! Since boost.mpi is just an abstraction layer over the native C MPI bindings, you can certainly mix the “raw” C MPI API in the same application with boost.mpi.

But I digress. Back to the original question…

The boost.mpi documentation has a nice tutorial explaining the basics. For most STL types containing intrinsic C types (e.g., the vector of integers example from above), you can just send/receive them and boost.mpi does the right magic behind the scenes. Something akin to:

For sending an receiving user-defined objects in an STL container, the Boost Serialization library can be used. You essentially add a “serialize” method on your class, which then allows the boost.mpi innards to pack and unpack STL container contents properly.

Refer to the boost.mpi and Boost Serialization documentation for more details.

7 Comments.

How does this work with one-sided, where remote unpacking must be done inside of MPI, rather than in the glue code between Boost::MPI and the MPI C bindings?
It would be good to show potential users how to use MPI datatypes in the official API to communicate STL objects, rather than to punt this problem to "magical" Boost and its antiquated MPI support.

boost.mpi doesn't work with one-sided. boost.mpi only "mostly supports MPI-1.1".
Since the MPI bindings are in C, they don't have good native support for things like the STL. That's why boost.mpi exists. I agree that its support is a bit antiquated at this point; it would be great if someone could update it to include a bit more than "most of MPI-1.1".
Additionallym such high-level concepts tend to not map well to hardware-offload scenarios. Meaning that one-sided operations of STL-like containers won't map well to hardware-accelerated message passing solutions because the hardware won't understand the (potentially) complex run-time created data layouts -- such code paths will likely fall through to software code paths (i.e., potentially slow).
Finally, don't forget that STL containers are allowed to hide their exact internal memory structure, and are even allowed to change it. For this reason, constructing an MPI datatype may not be sufficient for sending / receiving an STL container as elegantly as the boost.mpi solution because a) of potential memory allocation issues at the receiver, and b) the memory may well be non-uniform at either the sender or receiver (i.e., the expanded typemap may not be N times a single datatype typemap).

Regarding the documentation of Boost.MPI -- here's the version-invariant link: http://www.boost.org/doc/html/mpi.html
Another trick you can use is to replace the version in URL with word "release" (this shall point to the docs for the newest version), i.e., instead of http://www.boost.org/doc/libs/1_48_0/doc/html/mpi.html use http://www.boost.org/doc/libs/release/doc/html/mpi.html (notice how "1_48_0" was replaced with "release").
HTH!

I am not sure how much I like or dislike this answer. Boost is cool, more people should use libraries, *cough* MPI *cough*, but C++ has portability issues and while some things like the STL are well supported across compilers I have had issues with Boost and the Intel C++ compiler which is our default. The get the entire thing to play nicely you really need to use g++ which is fine just limiting.

That is a fair point.
Wearing my "MPI Forum" hat, I have to say that STL is a higher-level abstraction that lower-level communication libraries cannot possibly understand. MPI provides the building blocks for upper layer abstractions -- like the STL -- to take advantage of. boost.mpi is a good example of this, but as you mention, it has limitations.
Wearing my "I'm an MPI apps programmer" hat, I have to say, that sucks. :-(
FWIW, I think it would be great if someone developed a smaller, less all-encompassing MPI C++ class library (as compared to boost.mpi). Something that does a few things, and does them well -- that would be nice. Case in point: it took me 4+ hours to write this blog entry because I have to download and install boost first (which took several attempts to get right). Don't get me wrong -- boost is a very, very nice tool. But it is ginormous and more than most MPI developers may need. A small, purpose-built MPI C++ library (that includes the ability to send/receive STL containers) could be quite awesome.

Brock, you should really report those issues as bugs to your compiler vendor (Intel in case of ICC) as they indicate non-compliance with the ISO C++ standard which Boost adheres to strictly.
A vast majority (99%+) of the Boost libs relies on C++2003 or even C++1998. C++2011 has only begun to be considered recently and that usually in the context of providing a portable implementation of C++2011 features for C++2003 compilers such as Boost.Move (that being said, about 10 of Boost libraries have been incorporated into the C++2011 standard, so any quality issues with an implementation of Boost today indicate a potential issues in the future when C++2011 code and libs become more prevalent). It's 2012 now, nearly a decade, so non-compliance isn't really excused anymore.
That being said, we live in an imperfect world, so the libs are regularly tested -- see, for instance:
http://www.boost.org/development/testing.html#Introduction
http://www.boost.org/doc/libs/release/libs/math/doc/sf_and_dist/html/math_toolkit/main_overview/compilers_overview.html

Some of the individuals posting to this site, including the moderators, work for Cisco Systems. Opinions expressed here and in any corresponding comments are the personal opinions of the original authors, not of Cisco. The content is provided for informational purposes only and is not meant to be an endorsement or representation by Cisco or any other party. This site is available to the public. No information you consider confidential should be posted to this site. By posting you agree to be solely responsible for the content of all information you contribute, link to, or otherwise upload to the Website and release Cisco from any liability related to your use of the Website. You also grant to Cisco a worldwide, perpetual, irrevocable, royalty-free and fully-paid, transferable (including rights to sublicense) right to exercise all copyright, publicity, and moral rights with respect to any original content you provide. The comments are moderated. Comments will appear as soon as they are approved by the moderator.