Pages

Monday, October 25, 2010

This is beginning to have a very quixotic feeling to it. One of the lab consultants who's been away for five weeks on an internship stopped by today, and as I was updating him on my progress on the project it struck me how little I've actually accomplished in five weeks. True, I've overcome some major bariers and discovered some interesting things, but ultimately I'm still not doing distributed computing.

So far, my accomplishments include figuring out the SSH paswordless login with RSA encription, and getting the client machines to respond to unix commands from the host. Before (when I was using COPSSH), I kept getting an error saying that the 'env' command couldn't be found on the remote machine. 'env' is a really basic unix command, but it seems that it wasn't included in the version of Cygwin that was packaged with COPSSH. I resolved the issue by just installing Cygwin directly. The first time through, I didn't realize how enormous the full Cygwin set of packages is and ended up installing 6gb of mostly unknown material to a computer. I was gratified to see that SSH still works with just the minimal subset of Cygwin packages.

The problem I'm having now is even more frustrating, and gives credit to the recommendation from ReVolution not to mess with Windows clients. The issue is with the quoting of file paths, of course. Unix uses forward slashes and Windows uses backslashes; and in most programming languages the backslash is an escape character. Why oh why would Microsoft have chosen to ignore this reality? Unix and C were around and well established before they came on the scene with their dos-style paths... could that have done it explicitly to make it difficult to operate between platforms (and thus increase switching-costs?).

The crux of the issue is in the last line, right after the "\nws\bin\" bit. See that extra forward slash that follows? It seems thats inserted automatically by the stored function, regardless of what I enter as a parameter. Which is, incidentally, the following:

s=sleigh(

nodeList=c("coblab211-18.business.uc.edu"),

launch=sshcmd,

user="murphytj1",

scriptExec=scriptcmd,

scriptDir="'c:\\Program Files\\R-2.11.1\\library\\nws\\bin\\'",

scriptName="RNWSSleighWorker.py",

verbose=TRUE)

The double backslashes are what's required to overcome the escape character issue. The "scriptDir" is supposed to be the path to the file on the remote machine where the NWS client script is located. But despite the fact that I'm supposedly executing it with a windows style command (what the scriptExec option scriptcmd does), the function appends the forward slash. It is, I think, an error on the part of the writers of the NWS package. Now I'm trying to dig through their code to see if I can fix it; chances are slim.

Equally grim news on the Snowfall front; I can't even get sfCluster running on the host machine. Many many dependencies are apparently missing; starting with the gcc compiler (which I fixed). Even more frustrating, when I call make install, I get another error that looks like it was an issue with the coding:

Ok. Now call 'make install'.

sh-3.2# make install

Making/probing required folders...

mkdir -pv /usr/local/bin

mkdir -pv /usr/local/etc/sfCluster

mkdir -pv /usr/local/lib/sfCluster

mkdir -pv /usr/local/share/sfCluster

mkdir -pv --mode=700 /usr/local/var/sfCluster/tmp

mkdir: illegal option -- -

usage: mkdir [-pv] [-m mode] directory ...

make: *** [install] Error 64

sh-3.2#

What I should probably do is just drop both the macbook and the windows machines and try it on my Ubuntu laptop.