DESCRIPTION

This module is inspired in the IPC::PerlSSH module by Paul Evans. It provides Remote Procedure Calls (RPC) via a SSH connection. What made IPC::PerlSSH appealing to me was that

'no special software is required on the remote end, other than the
ability to run perl nor are any special administrative rights required;
any account that has shell access and can execute the perl binary on
the remote host can use this module'.

The only requirement being that automatic SSH autentification between the local and remote hosts has been established. I have tried to expand the capabilities but preserving this feature.

Provide Remote Procedure Calls (RPC). Subroutines on the remote side can be called with arbitrary nested structures as arguments from the local side.

The result of a remote call is a GRID::Machine::Result object. Among the attributes of such object are the results of the call, are the outputs produced in stdout and stderr, errmsg etc. The remote function can produce output without risk of misleading the protocol.

Services for the transference of files are provided

Support for writing and management Remote Modules and the transference of Classes and Modules between machines

An Extensible Protocol

METHODS ON THE LOCAL SIDE

The Constructor new

This function returns a new instance of an object. The object is blessed in a unique class that inherits from GRID::Machine. That is, the new object is a singleton. When later the machine object is provided with new methods, those are installed in the singleton class. The following example illustrates the point.

There are two GRID::Machine objects involved: $m (for a connection to a machine named orion) and $p (connection to a machine named beowulf) created at lines 7 and 13. Two subroutines with the same name one are installed on both machines (lines 10 and 16). As remote functions they don't collide since they are being executed in two different machines. As local methods they don't collide too since the method one of $m lives in a different namespace than the method one of $p. The remote functions are called in lines 11 and 17. The result of such call is a GRID::Machine::Result object. Such GRID::Machine::Result object describes the result of the RPC. It has attributes like:

results

A reference to an ARRAY holding the results returned by the call

stdout

The ouput produced in the remote stdout during the execution of the RPC

stderr

The ouput produced in the remote stderr during the execution of the RPC

etc.

Wherever is evaluated in a string context a GRID::Machine::Result object returns a string containing the output produced (to both stdout and stderr plus any specific perl error messages as in $@) during the execution of the RPC. When executed the former program will produce an output similar to this:

Exceptions

The constructor doesn't return on failure: It raises an exception if the connection can't be established. See the result of an attempt to connect to a machine when there is no automatic authentication:

Arguments of new

host

The host to connect. The user can be specified here. Also the port. I.e. it can be something like:

my $machine = GRID::Machine->new(host => 'casiano@orion:2048');

If host is the empty string:

my $machine = GRID::Machine->new(host => '');

a process executing perl in the local machine is open via open2 (no SSH call will be involved).

Instead of specifying the user, port and other ssh parameters here, the recommended way to work is to insert a section inside the /home/user/.ssh/config file:

...
# A new section inside the config file:
# it will be used when writing a command like:
# $ ssh gridyum
Host orion
# My username in the remote machine
user casiano
# The actual name of the machine: by default the one provided in the
# command line
Hostname orion.at.some.domain
# The port to use: by default 22
Port 2048
# The identitiy pair to use. By default ~/.ssh/id_rsa and ~/.ssh/id_dsa
IdentityFile /home/user/.ssh/orionid
# Useful to detect a broken network
BatchMode yes
# Useful when the home directory is shared across machines,
# to avoid warnings about changed host keys when connecting
# to local host
NoHostAuthenticationForLocalhost yes

command

This argument is an alternative to the host argument. Use one or the other. It allows a more specific control of the command executed.

It can be a reference to a list or a string. It fully specifies the command to execute.

Example 1: Using password authentication

The following example uses Net::OpenSSH to open a SSH connection using password authentication instead of asymmetric cryptography:

log

Relative path of the file where remote STDOUT will be redirected. Each time a RPC occurs STDOUT is redirected to a file. By default the name of this file is $TMP/rperl$LOCALPID_$REMOTEPID.log, where $TMP is the name of the temporary directory as returned by File::Spec-tmpdir()>, $LOCALPID is the PID of the process running in the local machines and $REMOTEPID is the PID of the process running in the remote machine.

err

Relative path of the file where remote STDERR will be redirected. Each time a RPC occurs STDERR is redirected to a file. By default the name of this file is $TMP/rperl$LOCALPID_$REMOTEPID.err, where $TMP is the name of the temporary directory as returned by File::Spec-tmpdir()>, $LOCALPID is the PID of the process running in the local machines and $REMOTEPID is the PID of the process running in the remote machine.

report

Relative path of the report file where the (remote) method remotelog writes. By default the name of this file is $TMP/rperl$LOCALPID_$REMOTEPID.report, where $TMP is the name of the temporary directory as returned by File::Spec-tmpdir()>, $LOCALPID is the PID of the process running in the local machines and $REMOTEPID is the PID of the process running in the remote machine. Set cleanup to false to keep this file.

wait

Maximum number of seconds to wait for the setting of the connection. If an automatic connection can't be established in such time. The constructor calls the is_operative function (see section "The Function is_operative") to check this. The default value is 15 seconds.

ssh

A string. Specifies the ssh command to be used. Take advantage of this if you want to specify some special parameters. Defaults to ssh.

sshoptions

An ARRAY ref or a string. Specifies options for the ssh command. See an example in which is a string:

Take into account that -MSys::Hostname takes place at a very early stage of the boot process and the functions will be exported to the main package. Therefore a use of the function hostname exported by Sys::Hostname inside a remote sub must be done as in this example:

remotelibs

An ARRAY reference. The referenced array contain the list of modules that will be loaded when bootstrapping the remote perl server. It is used to extend the GRID::Machine protocol. By default the following modules are loaded:

includes

A reference to an ARRAY of strings. Determines the "remote modules" that will be included when the remote Perl interpreter is started. The enumerated modules must be available on the local side. For instance, the following program loads the "Module":

A "remote module" resembles a module but contains only subroutines and uses. The functions are directly placed in the space name of the GRID::Machine object (just like the method sub does. See section "The sub Method"). Here are the contents of the "remote module" SomeFunc.pm:

A GRID::Machine::Result result object describes the result of a RPC. The results attribute is an ARRAY reference holding the result returned by the call. The other attributes stdout, stderr, etc. hold the respective outputs. See section "THE GRID::Machine::Result CLASS" for a more detailed description of GRID::Machine::Result objects.

The Algorithm of eval

When a call

$result = $machine->eval( $code, @args )

occurs, the code $code should be passed in a string, and is compiled using a string eval in the remote host:

my $subref = eval "use strict; sub { $code }";

Files STDOUT and STDERR are redirected and the subroutine referenced by $subref is called inside an eval with the specified arguments:

my @results = eval { $subref->( @_ ) };

Errors and Exceptions

If there are errors at compile time, they will be collected into the GRID::Machine::Result object. In the following example the code to eval has an error (variable $q is not declared):

The problem can be solved by redeclaring our $h in the second eval or changing the declaration at line 8 by use vars:

7 $machine->eval(q{
8 use vars qw{$h};
9 $h = [4..9];
10 });

Closures

One of the consequences of wrapping $code inside a sub is that any lexical variable is limited to the scope of the eval. Another is that nested subroutines inside $code will live in a (involuntary) closure. See the example:

The warning announces that later calls (in subsequent evals) to sub dumph can no longer reach $h (Other than trhough dumph itself). If you want lexical nested subroutines declare them through a reference:

This method sends code to the remote host to store it inside the remote side of the GRID::Machine object. Namely, the stored_procedures attribute of the remote object is a hash reference containing the stored subroutines. The string $code is compiled into a CODE reference which can be executed later through the call method.

The two first arguments are the name $name of the subroutine and the code $code. The order of the other arguments is irrelevant.

The subroutine name $name must be an identifier, i. e. must match the regexp [a-zA-Z_]\w*. Full names aren't allowed.

The following example uses compile to install handlers for the most common file-testing functions -r (is readable), -w (writeable), etc. (lines 8-15):

$ compile5.pl
Warning! Attempt to overwrite sub 'one'. New version was not installed.
one

The sub Method

Syntax:

$machine->sub( $name, $code, %args )

Valid arguments (%args) are::

politely => $politely

filter => $filter

around => sub { ... }

This method is identical to the compile method, except that the remote $code will be available as a (singleton) method of the $machine object within the local perl program. Therefore, two methods of two different GRID::Machine objects with the same $name are installed on different name spaces. See the example in section "The Constructor new".

The installed method $name can also be accessed as an ordinary function $name on the remote side. If a function with the same name already exists, the oldest prevails. See the call to function hi at line 15 in this example:

The filter Argument

By default, the result of a subroutine call is a GRID::Machine::Result object. However, for a given subroutine this behavior can be changed using the filter argument. Thus, the subroutine filter_results installed in lines 12-15 of the code below, when called returns the results attribute instead of the whole GRID::Machine::Result object. The subroutine filter_result installed in lines 17-20 returns the first element of the resulting list:

will apply the $filter subroutine to the GRID::Machine::Result object resulting from the call. The filter $filter must be a GRID::Machine::Result method and must return a scalar. The filter is executed in the remote side of the GRID::Machine.

The usage of the results and result filters can be convenient when the programmer isn't interested in the other attributes i.e. stdout, stderr, etc.

The around argument

A CODE reference. By default GRID::Machine produces a proxy representative in the local side for the sub being installed. The code of the proxy simply calls the corresponding sub in the remote side:

sub { my $self = shift; $self->call( $name, @_ ) };

You can substitute the proxy code by your own code using the around parameter. The proxy code receives the GRID::Machine object and the arguments for the remote side of the subroutine. See the following example:

When called, the squares function computes the squares on the remote side and adds one to each element in the local side:

$ perl around.pl
10 17 26

The makemethod Method

Syntax:

$machine->makemethod( $name, %args )

Valid arguments (%args) are::

politely => $politely

filter => $filter

around => sub { ... }

This method is identical to the sub method, except that it assumes the sub $name has been already installed in the remote side. The $name can be a fully qualified name, but the method call must use the short name. The following example produces a proxy method for the function reduce which is already available at the remote machine:

FUNCTIONS ON THE LOCAL SIDE

The read_modules Function

Searches for the specified modules Module:One, etc. in the local Perl installation. Returns a string with the concatenation of the contents of these modules.

For example, the line:

read_modules(qw(Parse::Eyapp Parse::Eyapp::))

returns a string containing the concatenation of the contents of all the modules in the Parse::Eyapp distribution. Modules are searched by name (like 'YAML') or by subcategories ('DBD::' means all modules under the DBD subdirectories of your Perl installation, matching both 'DBD::Oracle' and 'DBD::ODBC::Changes').

The Function is_operative

The syntax is:

is_operative($ssh, $machine, $command, $wait)

Returns true if $machine is available through ssh using automatic authentication and $command can be executed on the remote machine in less than $wait seconds. The following example illustrates its use:

The negative answer for the second execution is due to the fact that no command called chum is available on that machine.

If $machine is the empty string i.e. $machine eq '', it refers to a direct connection to the local machine and thus, it succeeds most of the time.

The Function qc

Prefixes the string passed as argument with the string #line $LINE $FILE where $LINE and $FILE are the calling line and calling file respectively. Used to provide more accurate error messages when evaluating remote code. See section "Errors and Exceptions".

THE TRANSFERENCE OF FILES

The put Method

Transfer files from the local machine to the remote machine. When no target directory is specified the files will be copied into the current directory (i.e. $ENV{PWD}). If targetdir/ is a relative path, it is meant to be relative to the current directory on the remote machine. It returns TRUE on success. See an example:

The get Method

Performs the reverse action of put. Transfer files from the remote machine to the local machine. When the paths of the files to transfer 'file1', 'filer2', etc. are relative, they are interpreted as relative to the current directory on the remote machine. See an example:

copyandmake copies (using scp) the files @files to a directory named $dir in the remote machine. The directory $dir will be created if it does not exists. After the file transfer the command specified by the copyandmake option

make => 'command'

will be executed with the arguments specified in the option makeargs. If the make option isn't specified but there is a file named Makefile between the transferred files, the make program will be executed. Set the make option to number 0 or the string '' if you want to avoid the execution of any command after the transfer. The transferred files will be removed when the connection finishes if the option cleanfiles is set. If the option cleandirs is set, the created directory and all the files below it will be removed. Observe that the directory and the files will be kept if they were'nt created by this connection. The call to copyandmake by default sets dir as the current directory in the remote machine. Use the option keepdir => 1 to one to avoid this.

LOADING CLASSES ONTO THE REMOTE SIDE

The modput Method

Syntax:

$machine->modput(@Modulenames)

Where @Modulenames is a list of strings describing modules. Descriptors can be names (like 'YAML') or subcategories (like 'DBD::' meaning all modules under the DBD subdirectories of your Perl installation, matching both 'DBD::Oracle' and 'DBD::ODBC::Changes').

The following example will copy all the files in the distribution of Parse::Eyapp to the remote machine inside the directory $machine->prefix. After the call to

The include Method

"The sub Method" permits the installation of a remote subroutine as a method of the GRID::Machine object. This is efficient when only a few subroutines are involved. However for large number of subroutines that procedure is error prone. It is better to have the code in some separated module. This way we can test the components on the local machine and, once we are confident of their correct behavior, proceed to load them onto the remote machine. This is what include is for.

Syntax of include

This call will search in the paths in @INC for Some/Module.pm. Once Some/Module.pm is found all the subroutines inside the module will be loaded as methods of the GRID::Machine (singleton) object $m. Code outside subroutines, plain old documentation and comments will be ignored. Everything after the markers __END__ or __DATA__ will also be ignored. The presence of the parameter 'exclude => [ qw( f1 f2 ) ]' means that subroutines f1 and f2 will be excluded from the process. Subroutine g1 will be renamed as min and subroutine g2 will be renamed as max.

Source after the __DATA__ or __END__ delimiters are ignored. Also, code outside subroutines (for example lines 13 and 14) and pod documentation are ignored. Only the subroutines defined in the module are loaded. See a program that includes the former remote module:

$ include.pl
user@remote.machine.es can do one
sub one
user@remote.machine.es can do two
sub two
user@remote.machine.es can do three
sub three

The use and LOCAL directives in Remote Modules

Two directives that can be used isinde a Remote Module are use and LOCAL:

A use Something pragma inside a Remote Module indicates that such module Something must be loaded onto the remote machine. Of course, the module must be available there. An alternative to install it is to transfer the module(s) on the local machine to the remote machine using modput (see section "The modput Method").

A LOCAL { code } directive inside a Remote Module wraps code that will be executed on the local machine. LOCAL directives can be used to massively load subroutines as in the example below.

Lines 9-17 are surrounded by a LOCAL directive and thus they will be executed on the local side. The effect is to install new methods for the GRID::Machine object that will be equivalent to the classic Perl file tests: -r, -w, etc. Inside a LOCAL directive the function SERVER returns a reference to the current GRID::Machine object (see line 11).

See a program that loads the former Remote Module. The call to include will load List::Util on the remote machine importing the sum function. Furthermore, methods with names sigma, r, w, etc. will be installed:

Specifying filters and proxys via #gm

We can include comments grid machine comments between the name of the subroutine and the open curly bracket to specify the options for the sub installation. The comments must start with #gm . See an example:

Sub last will return just the result instead of the full GRID::Machine::Result object. We have also substituted the proxy representative of method one using the around parameter. Consider the following script example:

THE GRID::Machine::Core REMOTE MODULE

The creation of a GRID::Machine object through a call to GRID::Machine->new implies the loading of a Remote Module called GRID::Machine::Core which is delivered with the GRID::Machine distribution. Another module that is being included at construction time is GRID::Machine::RIOHandle.

One of the final goals of the GRID::Machine::Core remote module is to provide homonymous methods per each of the Perl CORE:: functions. At present time only a few are supported.

The following functions defined in the Remote Module GRID::Machine::Core are loaded via the include mechanism on the remote machine. Therefore, they work as methods of the GRID::Machine object on the local machine. They perform the same operations than their Perl aliases:

Function qx

Similar to backtick quotes. The result depends on the context. In a list context returns a list with the lines of the output. In a scalar context reurns a string with the output. The value of $" on the local machine decides the register separator used. See an example:

The Protocol

The protocol simply consists of the name of the method to execute and the arguments for such method. The programmer - using inheritance - can extend the protocol with new methods (see the section "EXTENDING THE PROTOCOL"). The following operations are currently supported:

GRID::Machine::EVAL

Used by the local method eval

GRID::Machine::STORE

Used by the local methods compile and sub to install code on the remote side.

GRID::Machine::EXISTS

Used by the local method exists

GRID::Machine::CALL

Used by the local method call

GRID::Machine::MODPUT

Used by the modput method. A list of pairs (Module::Name, code for Module::Name) is sent to the remote machine. For each pair, the remote side writes to disk a file Module/Name.pm with the contents of the string code for Module::Name. The file is stored in the directory referenced by the prefix attribute of the GRID::Machine object.

GRID::Machine::OPEN

Used by the open method. As arguments receives a string defining the way the file will be accessed.

GRID::Machine::QUIT

Usually is automatically called when the GRID::Machine object goes out of scope

GRID::Machine::GPRINT

Most requests go from the local machine to the remote Perl server. However, this and the next go in the other direction. This request is generated in the remote machine and served by the local machine. It is used when inmediate printing is required (see section "Functions gprint and gprintf")

GRID::Machine::GPRINTF

This request is generated in the remote machine and served by the local machine. It is used when inmediate printing is required (see section "Functions gprint and gprintf")

GRID::Machine::CALLBACK

Used to implement callbacks

The SERVER function

The SERVER function is available on the remote machine. Returns the object representing the remote side of the GRID::Machine object. This way code on the remote side can gain access to the GRID::Machine object. See an example:

The stored_procedures method returns a reference to the hash containing the subroutines installed via the sub and compile methods. The keys are the names of the subroutines, the values are the CODE references implementing them. When executed the former program produces the list of installed subroutines:

$ accessobject.pl
tar
system
installed
getcwd
etc.

The read_operation Method

Syntax:

my ( $operation, @args ) = $server->read_operation( );

Reads from the link. Returns the type of operation/tag and the results of the operation.

The send_error Method

Syntax:

$server->send_error( "Error message" );

Inside code to be executed on the remote machine we can use the function send_error to send error messages to the client

This component will be loaded on the remote machine via the ssh link. The name of the handling method MYTAG must be the same than the name of the tag (operation type) used to send the request. Here is a client program using the new tag:

INMEDIATE PRINTING

Functions gprint and gprintf

When running a RPC the output generated during the execution of the remote subroutine isn't available until the return of the RPC. Use gprint and gprintf if what you want is inmediate output (for debugging purposes, for instance). They work as print and printf respectively.

Observe how the message 'Inside rmap!' generated at line 15 using print is the last (actually is sent to STDOUT in line 38). The messages generated using gprint and gprintf (lines 20, 26 and 29) were inmediately sent to STDOUT.

REMOTE DEBUGGING

To run the remote side under the control of the perl debugger use the debug option of new. The associated value must be a port number higher than 1024:

From now on you can execute almost any debugger command. Unfortunately you are now inside GRID::Machine code and - until you gain some familiarity with GRID::Machine code - it is a bit difficult to find where your code is and where to put your breakpoints. Future work: write a proper debugger front end.

THE GRID::Machine::Result CLASS

The class GRID::Machine::Result is used by both the local and remote sides of the GRID::Machine, though most of its methods are called on the remote side.

The result of a RPC is a GRID::Machine::Result object. Such object has the following attributes:

type

The type of result returned. A string. Fixed by the protocol. Common values are RETURNED and DIED.

stdout

A string containing the contents of STDOUT produced during the duration of the RPC

stderr

A string containing the contents of STDERR produced during the duration of the RPC

The result Method

Returns the first element of the list referenced by the results attribute This method is called when a GRID::Machine::Result object is evaluated in a Boolean context (i.e. bool is overloaded).

The Results Method

Returns the list referenced by the results attribute

The str Method. Stringification of a Result object

Returns the string made of concatenating stdout, stderr and errmsg. The Perl operator q("") is overloaded using this method. Thus, wherever a GRID::Machine::Result object is used on a scalar string context the str will be called.

THE GRID::Machine::Message CLASS

This class is used by both the local and the remote sides of the GRID::Machine. It implements the low level communication layer. It is responsible of marshalling the data.

The read_operation Method

Syntax:

my ( $operation, @args ) = $server->read_operation( );

Returns the kind of operation and the data sent by the other side of the SSH link.

Sends to other side of the link the type of the message and the arguments. It uses Data::Dumper to serialize the data structures.

REMOTE INPUT/OUTPUT

GRID::Machine objects have the open method. The open method returns a GRID::Machine::IOHandle object. Such objects very much behave as IO::Handle objects but instead they refer to handles and files on the associated machine. See a simple example:

from that moment on we can write in the file using the print and printf methods of GRID::Machine::IOHandle objects. You can see later in the former code how the diamond operator can be called to read on a remote file:

The fork method returns a GRID::Machine::Process object. The first argument must be a string containing the code that will be executed by the forked process in the remote machine. Such code is always called in a list context. The fork method admits the following arguments:

stdin

The name of the file to which stdin will be redirected

stdout

The name of the file to which stdout will be redirected. If not specified a temporary file will be used

stderr

The name of the file to which stderr will be redirected. If not specified a temporary file will be used

result

The name of the file to which the result computed by the child process will be dumped. If not specified a temporary file will be used

args

The arguments for the code executed by the remote child process

GRID::Machine::Process objects

GRID::Machine::Process objects have been overloaded. In a string context a GRID::Machine::Process object produces the concatenation hostname:clientPID:remotePID. In a boolean context it returns true if the process is alive and false otherwise. This way, the execution of line 21 in the program above:

21 print "Doing something while $p is still alive ...\n" if $p;

produces an output like:

Doing something while 5220:5230:some.machine:5234:5237 is still alive ...

if the remote process is still alive. The descriptor of the process 5220:5230:some.machine:5234:5237 is a colon separated sequence of five components:

The list of values returned by the child process. The forking code is always called in a list context.

status

The value associated with $? as returned by the remote child process.

waitpid

The value returned by the Perl waitpid function when synchronized with the remote child process. It is usually the value is either the pid of the deceased process, or -1 if there was no such child process. On some systems, a value of 0 indicates that there are processes still running.

errmsg

The child error as in $@

machineID

The logical identifier of the associated GRID::Machine. By default, 0 if it was the first GRID::Machine created, 1 if it was the second, etc.

The waitall method

It is similar to waitpid but instead waits for any child process.

Behaves like the wait(2) system call on your system: it waits for a child process to terminate and returns

The GRID::Machine::Process::Result object associated with the deceased process if it was called via the GRID::Machinefork method, or

The PID of the deceased process if there is no GRID::Machine::Process associated (it was called using an ordinary fork)

-1 if there are no child processes. Note that a return value of -1 could mean that child processes are being automatically reaped, as described in perlipc.

The async method

The async method it is quite similar to the fork method but receives as arguments the name of a GRID::Machine method and the arguments for this method. It executes asynchronously the method. It returns a GRID::Machine::Process object. Basically, the call

$m->async($subname => @args)

is equivalent to:

$m->fork($subname.'(@_)' args => [ @args ] )

The following example uses async to compute in parallel an approximation to the value of pi:

GRID::Machine::Process::Result objects

In a string context a GRID::Machine::Process::Result object produces the concatenation of its output to STDOUT followed by its output to STDERR. In a boolean context it evaluates according to its result attribute. It evaluates to true if it is an array reference with more than one element or if the only element is true. Otherwise it is false.

CALLBACKS

It may happen that the local machine has installed a useful set of modules that are not present on the remote side. It may be also imposible to transfer the modules to the remote machine using the mechanisms provided by GRID::Machine. In such situations -and many others - the callback mechanism can be helpful to achieve the task at hand.

The callback method provides a way to make a subroutine on the local side callable from the remote side. The ideas and implementation mechanisms used for callbacks is the work of Dmitriy Kargapolov (Thanks Dmitri!).

On success returns true, namely returns the address of the subroutine on the remote side that works as proxy of the localsub subroutine on the local side. Exceptions will be thrown in case of failure.

The following example shows a remote subroutine (lines 16-20) that calls a subroutine test_callback that will be executed on the local side (line 19). The call to the method callback at line 23 makes the local subroutine test_callback available from the remote side.

Callbacks and Namespaces

The callback subroutine is somewhat exported onto the remote side. That is, when transforming a local subroutine in a callback you can specify it by its full name (see line 24 below) but it is called from the remote side using its single name (line 18):

Recursive Remote Procedure Calls and Callbacks

The existence of callbacks opens the possibility of nested sets of RPCs and callbacks. The following example recursively computes the factorial of a number. The execution of recursive calls alternates between remote and local sides:

LIMITATIONS

Operating System

I will be surprised if this module works on anything that is not UNIX.

Opaque Structures

The RPC provided by GRID::Machine uses Data::Dumper to serialize the data. It consequently suffers the same limitations than Data::Dumper.

Namely, Opaque structures like those built by modules written using external languages like C can't be correctly transferred by the RPC system provided by GRID::Machine. An example is the transference of PDL objects (see PDL). In such cases, the programmer must transform (i.e. marshalling or project) the structure into a (linear) string on one side and rebuild (uplift) the (multidimensional) structure from the string on the other side. See an example:

Here the sdump method of PDL::IO::Dumper solves the problem: it gives a string representation of the PDL object that is evalued later to have the matrix data structure. When executed this program produces the following output:

Observe that variable $x is not modified. The only way to modify a variable on the local side by a remote subroutine is by result, like is done for $y in the previous example.

Limitations of the include Method

"Remote Modules" is the term used for files containing Perl code that will be loaded onto the remote Perl server via the incldue method or through the includes argument of new. These files can only contain

Subroutines. Since these subroutines are anonymous in the remote side, the only way to call them from the remote side is through the attribute stored_procedures of the SERVER object:

SERVER->stored_procedures->{subname}

use Module declarations. The Module must exists in the remote server. Furthermote module import arguments as in use Module qw{ w1 w2 w3} must be in a single line.

POD documentation

Variable declarations and variable initializations are ignored.

The include method parses Perl code. It is a heuristic one page length parser (72 lines at the moment of writing). It obviously can't parse everything. But works for most of the code restricted to the aforementioned limitations.

EXPORTS

When explicited by the client program GRID::Machine exports these functions:

However, it seems a bad idea to have unencrypted passwords messing around. It is much better to use asymmetric cryptography.

Using Automatic authentication: Asymmetric Cryptography

Set automatic ssh-authentication with the machines where you have an SSH account.

SSH includes the ability to authenticate users using public keys. Instead of authenticating the user with a password, the SSH server on the remote machine will verify a challenge signed by the user's private key against its copy of the user's public key. To achieve this automatic ssh-authentication you have to:

Generate a public key use the ssh-keygen utility. For example:

local.machine$ ssh-keygen -t rsa -N ''

The option -t selects the type of key you want to generate. There are three types of keys: rsa1, rsa and dsa. The -N option is followed by the passphrase. The -N '' setting indicates that no pasphrase will be used. This is useful when used with key restrictions or when dealing with cron jobs, batch commands and automatic processing which is the context in which this module was designed. If still you don't like to have a private key without passphrase, provide a passphrase and use ssh-agent to avoid the inconvenience of typing the passphrase each time. ssh-agent is a program you run once per login sesion and load your keys into. From that moment on, any ssh client will contact ssh-agent and no more passphrase typing will be needed.

By default, your identification will be saved in a file /home/user/.ssh/id_rsa. Your public key will be saved in /home/user/.ssh/id_rsa.pub.

Once you have generated a key pair, you must install the public key on the remote machine. To do it, append the public component of the key in

/home/user/.ssh/id_rsa.pub

to file

/home/user/.ssh/authorized_keys

on the remote machine. If the ssh-copy-id script is available, you can do it using:

The umask command is needed since the SSH server will refuse to read a /home/user/.ssh/authorized_keys files which have loose permissions.

Edit your local configuration file /home/user/.ssh/config (see man ssh_config in UNIX) and create a new section for GRID::Machine connections to that host. Here follows an example:

...
# A new section inside the config file:
# it will be used when writing a command like:
# $ ssh gridyum
Host gridyum
# My username in the remote machine
user my_login_in_the_remote_machine
# The actual name of the machine: by default the one provided in the
# command line
Hostname real.machine.name
# The port to use: by default 22
Port 2048
# The identitiy pair to use. By default ~/.ssh/id_rsa and ~/.ssh/id_dsa
IdentityFile /home/user/.ssh/yumid
# Useful to detect a broken network
BatchMode yes
# Useful when the home directory is shared across machines,
# to avoid warnings about changed host keys when connecting
# to local host
NoHostAuthenticationForLocalhost yes
# Another section ...
Host another.remote.machine an.alias.for.this.machine
user mylogin_there
...

This way you don't have to specify your login name on the remote machine even if it differs from your login name in the local machine, you don't have to specify the port if it isn't 22, etc. This is the recommended way to work with GRID::Machine. Avoid cluttering the constructor new.

Once the public key is installed on the server you should be able to authenticate using your private key

CONTRIBUTORS

Dmitriy Kargapolov (<dmitriy.kargapolov@gmail.com>) suggested, designed and provided an implementation for callbacks.

Eric Busto fixed a problem with is_operative hanging on systems that are in an odd state.

Alex White fixed bugs in modput and the SSH options.

Erik Welch fixed a bug in the (local) DESTROY method.

AUTHOR

Casiano Rodriguez Leon <casiano@ull.es>

ACKNOWLEDGMENTS

This work has been supported by CEE (FEDER) and the Spanish Ministry of Educacion y Ciencia through Plan Nacional I+D+I number TIN2005-08818-C04-04 (ULL::OPLINK project http://www.oplink.ull.es/). Support from Gobierno de Canarias was through GC02210601 (Grupos Consolidados). The University of La Laguna has also supported my work in many ways and for many years.

I wish to thank Paul Evans for his IPC::PerlSSH module: it was the source of inspiration for this module. To Alex White, Dmitri Kargapolov, Eric Busto and Erik Welch for their contributions. To the Perl Monks, and the Perl Community for generously sharing their knowledge. Finally, thanks to Juana, Coro and my students at La Laguna.