ARSC HPC Users' Newsletter 379, February 8, 2008

Contents

Call for Abstracts for the "Little Alaska Weather Symposium"

Abstracts for the Little Alaska Weather Symposium (LAWS) are due March 4. The May 12-13 LAWS at UAF provides a forum to exchange Alaska-specific operational forecasting and weather research data. For additional information:

CGI Library Available, C/C++

A library of CGI functionality is available for download. This library includes C and C++ functions, test programs and utilities for the common gateway interface (CGI) capabilities of Web servers such as Apache's httpd.

Development began in 1993, and the CGI library remains in use. With emphasis on security and stability, the library has been extensively tested in production environments. It is released under the BSD license.

NetCDF Conversion Example

[ By: Lee Higbie ]

I recently returned to a program I'd worked on several months ago. It uses data from other sources and seemed to be having data access problems. I decided to convert all those binary data files into a NetCDF file because:

ncview would quickly show gross errors

It would decouple my program from other programs whose output it uses

The next time I wondered about the data, ncdump -h would describe the data

Here are all the Fortran 90 statements required for the conversion program. I've stripped out much of the program but left what you'll need for creating a large NetCDF dataset. (BTW: UCAR, which developed NetCDF spells the name both with and without a capital N.)

This code shows all the statements for writing the grid-defining variables to a NetCDF dataset. The dataset is still open. An important note, each of the function calls above was actually the argument to a subroutine that printed diagnostic information if an error was detected.
The next steps were in a loop over all the binary files to add to this dataset. (do; open; read ....)

These are all the NetCDF calls necessary to create the file.
That program uses a series of namelists (samples there) to define the input data. If you want to use it as written, you will also need DebugClass.F95 (available there).

Compiler Stories

[ By: Kate Hedstrom ]

Those of us who develop software know how hard it is and know how many bugs are out there running around. What you may not realize is that the occasional bug isn't even yours, but that of the compiler vendor.

The optimizer is a good candidate for many troubles. Try compiling without optimization (-O0) to see if you get the same answer. Your code will run like a snail, but you need to find out if it's right before performance tuning. When I started using the ocean model I use, they liked to keep it all in one file, but had it in three for the Masscomp computer. Through much agony and effort, my colleagues had determined that they could aggressively optimize the number crunching routines, but not the I/O routines.

Automatic parallelizers can also do automatic bug insertion. I once spent a week tracking down what looked like an off-by-one indexing error in my plots to the auto-parallel compiler on the Alliant. I don't know why it took me so long to go back to serial mode for comparison.

Adding a print statement might change how the compiler views your code. I was having some odd behavior on the Connection Machine and I added a print statement to see what the compiler thought was the value of something. It then printed the value I expected - and the code ran perfectly!

Hidden corners of the language are less well tested than the commonly used parts. I was an alpha tester for g77 and had the most fun with a program using complex arithmetic. Most people use reals and integers so the complex data type had more lingering bugs.

Try other compilers if you possibly can. If I'm having problems, I try different optimizations, but I also love to try other compilers, if only for getting alternate warning messages. The NEC compiler was incredibly picky and verbose, which is great when you're trying to track something down. Also, many compilers have some sort of bounds-checking capability which can be worth trying.

If think you smell a compiler bug, what next? You can find a way to work around it and get on with life, like my friends did on the Masscomp. If it is on one of the ARSC systems, you can report it to consult, who will get it reported to the compiler vendor. The vendors will be more or less responsive, depending, but most of my reported IBM bugs have been fixed. Finding a way to get your work done in the mean time is usually a good idea, though.

If you report it, the shorter the code the better. No one wants to wade through 200 source files in 15 directories, using umpteen external libraries. Get an example code demonstrating the problem into one short file if possible. I'm not saying this is easy, but it will be easier for you who know the code than for the compiler people. As you shorten the code, keep a prior version demonstrating the trouble because at some point in your whacking the trouble will likely disappear. Go back to the previous version and whack in a different area. Once you get it "short enough", write up the expected results, the actual results, and how to compile and run your code snippet.

Quick-Tip Q & A

A:[[ Do vi or VIM offer any way to repeat commands issued at the colon
[[ prompt (i.e., "ex" commands)? For instance, I might want to execute
[[ the following search and replace on one line and then hop to another
[[ line and do it again:
[[
[[ :s/-/=/g
[[
#
# Thanks to Ryan Czerwiec
#
I'm not aware of such a command, and the vi/vim help doesn't list one.
If you happen to want to change every instance in the file, you can hit
Ctrl-g to see how many lines there are, then preface your search command
with the range of line numbers desired:
:1,100 s/-/=/g
for instance. I believe there's a quicker shorthand than looking up the
last line number if you do want the entire file. If you want to check
each instance rather than do a blanket search-and-replace, use gc
instead of just g as the option, and it will ask you for confirmation at
each instance of the search string. If there are too many instances for
this to be practical, I usually just highlight the command with my mouse
and use the cut-and-paste capabilities of the mouse buttons to
re-execute with a click. As a bit of a shorthand, you can eliminate the
search strings, and it will assume they are the same if you repeat your
search:
:s g
:101,200 s g
#
# Thanks to Lee Higbie
#
One easy way to repeat vi or vim commands:
Once you type your command, ":s/-/=/g," for example, copy it to your
terminal's buffer, then type the Enter key. Move to the new location,
paste from the terminal and type the Enter key and so on.
Using a local machine or its terminal program to an HPC machine running
the editor:
Type the command (don't type the Enter key yet) Use the mouse to
highlight the command and copy it (Ctrl+c, Cmd+c, or mouse click
depending on your setup) Type the Enter key move the cursor to the
new line you want to edit Paste (Ctrl+v, Cmd+v, or mouse click)
Type the Enter key
This works because the first character in the copy buffer is the first
character of the command, the colon. When you paste from the buffer,
vi or vim interpret it just as though you had typed the command.
#
# Thanks to Kate Hedstrom
#
Yes, you can. ":%" maps to all lines. So you can do:
:%s/^C/!/
to convert old Fortran comments to new ones. In fact, I've got a .exrc
file with "map g :%" in it (from an ancient Sun starting Unix guide)
so that what I type is "gs/^C/!/" for global substitution.
#
# Thanks to Oralee Nudson
#
Type an "&" while in escape mode to repeat the last "s" command.
#
# Editor's solution
#
In VIM, the register ":" contains the most recent ex command. The
command "@:" executes the register ":". So, to answer the question,
simply hop to the next line and type, "@:".
As another example, an easy way to replace the first 16 "-'s" with
"='s" on the same line would be this sequence:
:s/-/=/
15@:
Q: I am currently debugging my code by submitting jobs to the "debug"
queue on midnight. To properly test my PBS scripts, I need to submit
a handful of jobs at a time to see if each job will pick up where the
previous job left off. I often find a bug early in the job sequence,
resulting in a handful of test jobs still queued without any real
purpose. Rather than waiting for each of these jobs to finish,
I have been using the "qdel" command to cancel them. For example:
% qdel 123456 123459 123479 123480
Depending on the number of debug jobs still queued, it can become
tedious to type or copy and paste each of these job IDs for every
iteration of my debugging efforts. Is there an easier way to delete
all of my jobs in the debug queue?

The University of Alaska Fairbanks is an affirmative action/equal
opportunity employer and educational institution and is a part of the University
of Alaska system.
Arctic Region Supercomputing Center (ARSC) |PO Box 756020, Fairbanks, AK 99775 | voice: 907-450-8602 | fax: 907-450-8601 | Supporting high performance computational research in science and engineering with emphasis on high latitudes and the arctic.
For questions or comments regarding this website, contact info@arsc.edu