Contents

Converting Byte Order of Binary Data Files on the Cray XD1

[ Thanks to Ed Korkven of ARSC for this article, the first in a Series on Porting Codes From the Cray X1. ]

In anticipation of the September 30 decommissioning of Klondike, ARSC's Cray X1, ARSC is urging Klondike users to begin planning for that transition. One of the major concerns that users may have is the portability of data files produced on the X1. Text files (i.e. those created with the Fortran OPEN option FORM='FORMATTED') are not a problem since by default the X1 creates ASCII text files and any machine to which users would port those files will read ASCII text files transparently, perhaps with minor end-of-line differences for non-Unix machines. But what about binary data files (i.e. those created with the OPEN option FORM='UNFORMATTED')?

The X1 differs from many earlier Cray machines in that by default it uses 32-bit IEEE standard floating point with two's complement arithmetic. This is an advantage since this format is commonly used on most other platforms likely to be targets for porting from the X1. (We would appreciate hearing of any exceptions.) Even when porting to another IEEE platform however, there are some issues to consider. In this and following newsletters we will discuss strategies for doing these ports.

In this issue, we will begin to take a look at the "byte order" of the contents of a binary data file. The byte order of a machine refers to the order in which it stores multi-byte data items in memory, and then in data files. Klondike is a "big-endian" machine. If moving to another big-endian machine, byte order will not be a consideration. One notable CPU set with opposite byte order however, is the x86 family. This includes the AMD chips that are prevalent in many machines, notably, Cray's XT3 and XD1 platforms. Reading unformatted X1 files on such computers will require some kind of special handling either on the X1 or on the target.

ARSC's XD1 machine has a very handy feature built in to its Portland Group Fortran compiler that automatically converts big-endian to little-endian and vice versa when UNFORMATTED data is read or written. Here is an example. The following is an X1 program that writes a binary data file called "output.dat".

If we compile the reader program with: pgf90 -o reader reader.f90 we get no "match" messages during execution because the X1 big-endian output is read as little-endian values. Recompiling with:

pgf90 -byteswapio -o reader reader.f90

gives the result we hoped for:

Integer*4 matches.
Real*4 matches.
Integer*8 matches.
Real*8 matches.

So the Portland Group compiler provides a simple solution to byte order differences between big-endian and little-endian machines. And this example will hopefully motivate X1 users to start thinking about the porting process -- for data as well as code. ARSC staff will be happy to assist Klondike users during the transition.

Handel-C Programming Language Class, for FPGA Applications

ARSC will offer training for the Handel-C programming language and DK toolkit from Celoxica. The class will use Celoxica's RC-10 reconfigurable computing boards.

Class schedule:
Tuesday July 25 through Friday July 28
9:00 am - 5:00 pm each day
West Ridge Research Building (WRRB) room 104

This is a class for electrical & computing engineers and others interested in using field-programmable gate arrays (FPGAs) for a variety of applications. Celoxica's Handel-C is one of several programming toolkits that ARSC is investigating this summer as part of an ongoing study of high-performance reconfigurable computing.

Space is limited. Please email Greg Newby (newby AT arsc.edu) to register. There is no fee to attend this training.

Parallel Performance Evaluation Tools Workshop

Quick-Tip Q & A

A:[[ I have an existing library written in C/C++ that I would like to
[[ call from a python script. Is there a way to call this existing
[[ code without reimplementing the code in python?
#
# Thanks to Brad Chamberlain:
#
While I don't have firsthand experience with it, I do know that
the Babel package out of LLNL is designed to support C/C++/Python
interoperability (as well as Fortran and Java). For more information,
see:
http://www.llnl.gov/CASC/components/babel.html
#
# Editor's response
#
SWIG (http://www.swig.org/) is a tool which connects C/C++ to high
level languages such as Python, Ruby, Java, Tcl, PHP. The steps are:
1) Build an interface file for your code so SWIG knows what portions
of the code should be available in the python modules.
2) Run swig on the interface file to create a wrapped C file and the
corresponding python module.
3) Compile the wrapped C file and other C source files.
4) Build a shared library using the object files.
There is a basic tutorial here:
http://www.swig.org/tutorial.html
Q: I want to search for certain keywords in a monstrous text file.
But for each "hit," I want to see, not just the line containing the
word, but a few lines **preceding** it as well. I think I could
write a perl script to do this, but is there another way?

The University of Alaska Fairbanks is an affirmative action/equal
opportunity employer and educational institution and is a part of the University
of Alaska system.
Arctic Region Supercomputing Center (ARSC) |PO Box 756020, Fairbanks, AK 99775 | voice: 907-450-8602 | fax: 907-450-8601 | Supporting high performance computational research in science and engineering with emphasis on high latitudes and the arctic.
For questions or comments regarding this website, contact info@arsc.edu