Commands are implemented as agents and can be run in any node. A command is defined as a description and an agent type and is registered with a unique name.

package org.agilewiki.jasocket.commands;

public class Command {

private String description;

private String type;

public Command(String description, String type) {

this.description = description;

this.type = type;

}

public String description() {

return description;

}

public String type() {

return type;

}

}

The Commands actor is the base class of the registry for commands. Commands are registered when the actor is initialized, so the set of commands is immutable. A convenience method is also provided for registering the command's agent factory.

Agents are initialized with their own mailbox. And unless the agent overrides the async method, their mailbox will be an async mailbox.

Agents which have been shipped to another node are initialized with the AgentChannel that delivered them as their parent. Otherwise the parent is the AgentChannelManager, in which case the isLocal method returns true.

The isLocalAddress method can be used to determine if a given node address is for the node where the agent is currently executing.

Note that in the above, the second argument passed by main to the create method is null. This means that the console will not support user interrupts. Unfortunately, user interrupt processing requires sun JRE-specific methods. So we handle user interrupts in a separate class, IntCon. If using a different JDK, you will need to replace IntCon entirely rather than subclass it:

The Node class provides a default configuration for a node in the cluster and also has a main method for running a simple node, e.g. a node without a console or any initial servers. Configuring a node is a matter of writing a subclass of Node, overriding the various defaults set by Node and adding servers that be started when the program runs. SSHServer and HelloWorld are examples of classes with a main method that creates a node and then create a SSHServer or HelloWorld server respectively.

Here then is the first part of the Node class, including all its public methods:

Note in particular the call to the Node constructor in the main method. The argument provided to the constructor is the number of threads to be created. A 100 is probably excessive. In addition to the threads needed to run the application logic (figure 1 per hardware thread plus 1 for every blocking actor), JASocket requires 1 thread for its Timer and an additional thread for each channel.

The startup method is also worth noting, as it can be called from main to start a server.

Node Directory

Every node has its own unique directory. To ensure this, the directory is given a name based on the cluster port:

This max size is then set for all the sockets, and also specifies the size of the ByteBuffer to be used.

Multicasting

Multicasting is used for discovery and a number of items need to be specified:

protected void startDiscovery() throws Exception {

new Discovery(

agentChannelManager,

NetworkInterface.getByInetAddress(InetAddress.getLocalHost()),

"225.49.42.13",

8887,

2000);

}

The most interesting is the network interface. Generally you can use the network interface which supports the local host IP address. But if you have multiple internet connections, e.g. more than one eithernet card, or if you are using a VPN, you may need to specify something different.

The IP address given above is just an arbitrary IP address in the range of multicast IP addresses. Each IP address (and port number) specifies a multicast group. You must specify a different address or port for each cluster.

The last parameter passed to the Discovery constructor is the delay between broadcasts, in milliseconds. When a node first starts up it sends out a UDP multicast packet. This packet is then rebroadcast repeatedly after the given delay.

To disable discovery, simply override this method with one which does not create an instance of Discovery.

Keep Alive

To use keep alive's, just call the startKeepAlive method on the agent channel manager:

protected void startKeepAlive() throws Exception {

agentChannelManager.startKeepAlive(10000, 1000);

}

The first parameter specifies how long a socket can be idle before being closed, in milliseconds.

The second parameter specifies the delay between keep alive messages, in milliseconds. These are only sent on channels which are inactive.

Keep alive's are important for detecting network failure, but may not be helpful when the entire cluster is running on a single computer. To disable their use, simply override this method with one which does not call startKeepAlive on the agentChannelManager.

Operator Authentication

Operator names/passwords can be authenticated. But straight out of the box, JASocket does not provide any authentication. Note that if you do provide authentication, operator names must not be allowed to contain spaces.

To run JASocket you will need the JAR files for compatible versions of JActor, JID and JASocket. Download these projects from here and extract the jar files to a common directory, in our case c:\jasocket. You will also need jar files from slf4j, sshd, joda-time and jline.

Next you need a few shell scripts. Here are some which work with windows:

runnode.bat - provides the classpath, including the simple jar which displays log messages:

throughputTest - Benchmark throughput between this node and another node

to - Send a command to another node

who - List all user names/node/logonTime/commandCount/idleTime/ssh

write - send a message to a user

2>

Enter the channels command to see what nodes it is connected to:

2>channels

10.0.0.2:8880

3>

The to command is used to send a command to another node in the cluster. Enter the command "to 10.0.0.2:8880 channels" to see what nodes the second node is connected to (adjusting the IP address as needed for your compute, of course):

The same exercise can be repeated using sshnode.bat instead of consolenode.bat. On windows, the PuTTY program can be used to open a session. (Again, when used straight out-of-the-box all user names and passwords are valid. Only the name may not contain a space.)

JASocket is a lock-free, scalable and robust server framework with no single point of failure. Servers are run on a cluster of nodes and interact with other servers using mobile agents, which reduces the number of messages and thus reduces the overall system latency. Administration is handled via ssh.

Latency for agents sent between 2 JVM’s on the same machine (round trip): 84 microseconds. Throughput for agents sent between 2 JVM’s on the same machine: 137,000 per second.

JASocket is licensed under LGPL. The project page can be found here. You can also download both the source and the JAR files here. Maven users will find also this project in the Central Repository.

Why Agents?

JASocket supports both requests/responses (2-way messaging) and notification events (1-way messaging), where requests and notification events are mobile agents and responses are actors. (The difference is that actors do not receive a Start request when deserialized on arrival.) The idea is that an agent can interact with any number of other actors when it arrives at its destination and the actor sent as a response can hold the requested information or serve as a smart proxy for subsequent requests. This use of agents and actors in place of simple messages then holds significant potential both for reducing the volume of traffic and for reducing the number of message exchanges, thereby increasing the throughput and reducing the latency of the cluster as a whole.

Will JASocket run in the Cloud?

Administration is via ssh, which is compatible with use in a cloud. The JAConfig project builds on JASocket and provides basic management capabilities, but has no dependencies on any Cloud API.

How does a Node learn about other Nodes in the Cluster?

Nodes periodically send a multicast UDP packet containing its cluster port number. When a node receives one of these packets, it checks to see if it already has a connection. If not, it opens a connection and sends a SetClientPortAgent that will associate its server port number with the connection in that remote node. Connections then are bi-directional and every node is connected with all other nodes, so long as all nodes are under a common NAT (if any)--this technique is not compatible with port mapping.

Does JASocket detect Network Failures?

Using TCP, a connection to another node is closed (with an EOF) when the other node terminates normally. But a network failure,a cable disconnection for example, is not detected even when the TCP KEEP ALIVE option is enabled. To detect network failure, a KeepAliveAgent is sent on all connections which have not sent any messages/actors in the last N1 milliseconds and connections are closed which have not received any actors/messages in the last N2 milliseconds. (N2 is generally a multiple of N1 and must be large enough to accomodate both Java garbage collection and any periodic resource starvation imposed by the [MS Windows] operating system.)

Does JASocket use Nagle's Algorithm?

Nagle's algorithm is used to improve TCP socket throughput, but also increases latency. It is enabled by default and is turned off through the use of the TCP NODELAY option--which JASocket does. Rather, JASocket does dynamic buffering, aggregating outgoing traffic until either the output buffer is full or the actor has no pending input. JASocket latency between nodes running on the same machine is 42 microseconds, including serialization and deserialization.

How are Servers located?

Servers running on a node are registered on that node with a unique name. The names are shared across nodes, when a server is registered (startup), when a server is unregistered (shutdown), and when a new connection is made between nodes. And when a connection is closed, all the names of all the servers residing on the (now unreachable) remote node are dropped.

Server names can be used in place of node addresses when shipping an agent to another node. If the same name is used to register resources on multiple nodes, an arbitrary choice is made.

When deserialization and reserialization are reasonably fast, their use to make deep copies of data structures becomes a reasonable approach. And Jid provides the CopyJid request, which is supported by all Jid actors. The GetSerializedBytes request is similar, in that it returns a byte array holding the serialized data of a Jid actor, and it too is supported by all Jid actors.

Making copies of data structures is important in a multithreaded application when it can be used to reduce the number of messages sent between threads. Conversely, being able to add a copy of a Jid actor to a collection may be even more useful. This is done by first getting the byte array of a Jid's serialized data and passing it in one of several requests which then create a copy of that Jid and add it to their collection. These requests include SetActorBytes for RootJid, ActorJid and UnionJid, IAddBytes for BListJid and KMakeBytes for BMapJid.

All Jid objects have the same superclass, Jid, which in turn is a subclass of JLPCActor, which means that all Jid objects are actors.

So far, we have not given any examples of a Jid object initialized with a Mailbox, which means that none of the Jid objects shown are able to send or process messages. But initializing a Jid object with a Mailbox is easy to do and most of the methods in the JID API have corresponding Request classes. Also, the Jid objects in a Jid tree structure will always share the same mailbox, so an application Jid never needs to send Requests to the Jid objects in its tuple--it can just call their methods directly.

In the code below we create a RootJid with a JidString set to "Hello world!", serialize it and then deserialize it. Many of the method calls shown earlier have been replaced with request messages to illustrate their use. However, the serialization and deserialization logic still uses method calls, which means that thread safety is the responsibility of the application developer for these operations. (Thread safety can always be achieved by performing these operations within an actor which uses the same mailbox as the Jid Actor.)

BMapJid is the base class for balanced tree maps which, like bListJid, provide for super-fast incremental deserialization and reserialization. BMapJid has 3 subclasses, IntegerBMapJid, LongBMapJid and StringBMapJid, which support Integer, Long and String keys respectively.

BMapJid<KEY_TYPE, VALUE_TYPE> is a collection of MapEntry objects, where MapEntry holds a key/value pair. BMapJid is effectively a sorted list of MapEntry objects, with fast indexing supporting the same methods as BListJid exception only the iAdd and iAddBytes methods are not supported. But access by key is also supported. These additional methods include

MapEntry<KEY_TYPE, VALUE_TYPE> getCeiling(KEY_TYPE key) - Returns the MapEntry with the smallest key that is greater or equal to the given key, or null. And

MapEntry<KEY_TYPE, VALUE_TYPE> getHigher(KEY_TYPE key) - Returns the MapEntry with the smallest key that is greater than the given key, or null.

BMapJid objects are created using a registered factory object. As a convenience, JidFactories registers 24 such factory objects, though it is easy enough to define register additional factory objects using the IntegerBMapJidFactory, LongBMapJidFactory and StringBMapJidFactory classes.

For high-performance, array-backed data structures are generally recommended. On the other hand, inserting into a large array isn't the fastest thing. BListJid uses small arrays (max size is 27) in a balanced tree structure to support fast updates and super fast incremental deserialization/reserialization.

As with other Jid classes, a registered factory object is mandatory. The JidFactories class registers 8 different types of BListJid, one of which is a list of integers.

The iAdd method creates an inserts a new Jid object at a given location, where 0 is the first location, 1 is the second, -1 is the last location, -2 is the next-to-last location, etc.

BListJid supports only homogenous lists, where all the entries are of the same class. The advantage is that the serialized data is smaller than it would otherwise be and performance is a bit better as well. But sometimes we need a list of homogenous objects. This can be achieved with ActorJid, which is a superclass of RootJid and which can hold any object that subclasses Jid.

Being able to constrain the types used in a data structure can be important, and this is one of the advantages of using UnionJid instead of ActorJid. It also results in serialized data that is a bit more compact and a bit faster to deserialize. It also supports recursive types, which is what we will be looking at in this next example.

In the above code we define the type union1, which can hold either a StringJid or a Jid of type unions. And the type unions is defined as a BListJid whose elements are of type union1. We then proceed to create a list whose first element is a StringJid with a value of "a" and whose second element is a list whose first element is a StringJid with a value of "b".