Ever since Google presented Wave – a collaboration and communication platform – to the world during Google I/O 2009, the Internet has been a buzz with all things Google Wave.

As with every new technology, controversial use case discussions quickly dominated the technology and geek blogosphere.

Google Wave is constantly evolving. Before we start: To stay in touch I kindly ask you to follow me on Twitter (@24z).

One of the key aspects of Google Wave is its federation architecture. Much as email is distributed across hundreds of thousands of distributed servers, implementing standards as POP3, SMTP and IMAP, Google Wave servers are designed from the ground up to support federation, too.

Not only does this allow third parties, including open source initiatives, to come up with their very own implementation of Google Wave servers, it also facilitates (corporate) security, scalability, innovation and, I’m sure sometime soon, hosted offerings.

Given that federation is such a key aspect of Google Wave, it comes as no surprise, that a recent article I’ve published over at The Next Web unveiling Google’s plans to open their developer sandbox environment, quickly ended up on Techmeme’s front page.

This blog post covers my journey in getting federation between my server and the Google Wave developer sandbox to work.

Many aspects of Wave federation are still very beta – some folks on the wave-protocol list suggested they are even pre-alpha – so details of my findings are guessings and assumptions. Feel free to correct me where I’m wrong.

I anyway hope, this contributes a bit to the passionate Google Wave scene and hope to hear back from you.

What does “federating to sandbox works” mean?

When Google introduced Wave, invites where given to the 4000+ Google I/O attendees and in a second roll-out phase to more developers. Those invites allowed users to use the Google Wave infrastructure at http://www.wavesandbox.com. Usernames for this so called Wave sandbox environment are <your alias here>@wavesandbox.com.

Later this year a second environment, the Google Wave preview, invited 100.000+ additional accounts. This environment is at http://www.googlewave.com and users are <your alias here>@googlewave.com.

The Wave sandbox and the preview environment are two totally separate infrastructures with no bridge between the two. As of this writing, you can federate with sandbox only. googlewave.com has not been opened for federation, yet.

Successfully setting up federation at the time of this writing means, that you

run your own Google Wave server, very likely based on the FedOne reference implementation provided as open source by Google

are able to create Waves either via the GWT based web user interface for the Wave sandbox or through the command line client for your own server, add a participant from the respective other environment and see these Waves on both ends

see live typing (character-by-character) when modifying a Wave on Wave sandbox

A typical sessions will look like this, click the image to enlarge:

(Note: The image has been assembled in Photoshop to show you both, the GWT based Wave sandbox client and the FedOne command line client. The session, however, is a real-world federated session.)

Required reading before you start

Though you very likely read all of the below more than once, I’d like to point you to my list of required reading for the sake of completeness:

I highly encourage to print out all of the above and thoroughly go through it while you’re away from the distractions of your desktop. You will likely revisit the documents time and again anyway, but it helped me a lot to take a break and give them 100% of my attention.

You should also revisit the pages frequently, as they constantly undergo changes influenced by community feedback. As an example the wiki article on certificates has been enhanced while I’ve been working on this post, making many aspects more clear as in the initial version.

My setup

Theoretically federation should work on Windows and any flavor of Unix/Linux, including Mac OS X (Snow Leopard). I got it working on:

Part I: Installing the basics, getting FedOne up and running

The installation instructions provided by Google are actually quite accurate. I had no trouble getting this part done. I added two steps which have not been on Google’s wiki page though: Turning off Openfire’s pubsub service and the group chat features.

Here’s a quick rundown:

As described in “My Setup” above I’ve used a cloud server to set up federation. Most cloud servers come with a static IP address. To get federation working it seems essential that you’ve got access to a DNS server in order to set up the correct records that point to your server. I’ll go into more details later. For now it’s only important that you’ve got at least a domain pointing to your server. In my case it’s wave24z.com and the DNS A record points straight to my cloud servers IP. Wherever you read wave24z.com in the following paragraphs, you’ve got to substitute with your own domain name.

There has been quite some discussion on the wave-protocol list whether Sun’s Java SDK or the Open Java SDK should be used. Some group members pointed out potential incompatibility issues with certificates issued by CAcert and Sun’s Java SDK. I do not use CAcert and cannot personally confirm this. As Google recommends using Sun’s Java SDK go ahead and install it by running apt-get install sun-java6-jdk sun-java6-fonts

Once the wizard has completed setting up Openfire, you have to restart the Openfire server. If you forget to do so, you will not be able to log in. On the command line runsudo /etc/init.d/openfire restart

Point your browser to http://<yourdomain>:9090 and log in. For the username use admin and not your email address. Enter the password you’ve chosen during setup. Trying to log in with your email address as the username will fail. (This turned me nuts, so I thought I emphasize it here. The initial version of Google’s wiki did not have the bold note on using admin instead of your email address.. .)

Once logged in, you’ve got to modify some settings:
– In Server Settings > External Components > Service Enabled select Enabled keep 5275 for Port and enter a Default shared secret. This can be an arbitrary string.
– Further down in Allowed to Connect select Whitelist in Subdomain enter wave and in Shared secret enter another arbitrary string. I’ve used the exact same secret for both (Default shared secret and Shared secret for the subdomain), but I’m not sure whether this is required.
– In Server Settings > Security Settings > Server Connection Security select Custom and make sure that Available is active for Server Dialback and Optional is selected for TLS method. Also check Accept self-signed certificates. Hit Save Settings. Note: The radio button might jump back to Optional. Don’t worry. This is ok.
– In Server Settings > Compression Settings > Server Compression Policy select Available.
– In Server Settings > File Transfer Settings select Disabled.
– In Server Settings > Registration & Login select Disabled for the top three options, Inband Account Registration, Change Password and Anonymous Login.– In Server Settings > Logs enable debug logging by selecting Enabled on the Debug tab.
– In Users/Groups > Users create a new user. Give it a simple one-word username. I chose ralf. I’m not 100% sure whether creating a user is required to have federation working. It’s just what I did and I thought I let you know.
– In Server Manager > System Properties scroll to the bottom, enter xmpp.pubsub.enabled into the Property Name field and false for the Property Value field. Hit Save Property. This turns off Openfire’s pubusb service which is known to interfere with how FedOne discovers XMPP items.– In Group Chat / Group Chat Settings delete the conference domain.

Next grab the wave-protocol source code, which includes the FedOne Prototype Wave server and the command line clienthg clone https://wave-protocol.googlecode.com/hg/ wave-protocol

Build the projectcd wave-protocolant

The build should take approx. 45 seconds. If you’re seeing some warnings while building that’s okay. If the build fails stating something about compliance and Java version, make sure that your system is using the version of Java installed at the very beginning. If not, change it viaupdate-alternatives –config javaand making the appropriate selection.

These instructions are actually more complex than the process itself. The entire setup only takes between 10 – 15 minutes of your time, depending mostly on the bandwidth for downloading the bits and your server’s processing power.

Part II: Domain Name System (DNS) setup

The federation wiki article goes into a great amount of detail as to how important the correct DNS setup is and why it matters. While theoretically FedOne should fall back to alternative mechanisms if the lookups defined by the XMPP standard fail, I could not get federation to work without setting up the correct SRV records.

The exact way to set up DNS records depends much on the system you’re using. I’ve decided to use DynDNS for all my DNS needs long time ago and am pretty happy with it. It’s reliable, propagates fast and by taking away DNS services from my cloud servers, I have a clear line for separation of concern which makes troubleshooting a lot easier.

Please note that I’ve created A records – these are “regular” DNS records as they get created when you’re registering a domain with your service provider – not only for the main domain but also for the subdomains wave.wave24z.com and pubsub.wave24z.com. I’m not sure whether the pubsub subdomain is a requirement. It’s not mentioned in Google’s wiki. I just recognized error message in FedOne’s console log pointing to problems finding pubsub.wave24z.com so I thought it might not hurt. Turning Openfire’s pubsub services off as indicated above, should render adding the pubsub subdomain obsolete.

The MX records are there to allow emails to be send to addresses @wave24z.com. DynDNS offers mail forwarding for each domain operated via their dynamic DNS servers, so I don’t have to run an SMTP daemon on my cloud server. The steps required on your side might vary.

You should at least make sure that you can receive email at webmaster@<yourdomain>. This address is used by StartSSL to verify you’re the owner of your domain, when we’re going to create certificates later.

The last two entries are the SRV records which allow other Wave servers to discover the address and port FedOne is running on. Please note that I’ve configured them for both, the main wave24z.com and for the subdomain wave.wave24z.com. Essentially these records tell XMPP servers looking for the XMPP services running on _xmpp-server._tcp.wave24z.com that the service is available on port 5269 at wave24z.com.

Once you’ve configured DNS settings and let them propagate, you should verify that they resolve correctly by submitting

dig +short -t SRV _xmpp-server._tcp.wave24z.com

and

dig +short -t SRV _xmpp-server._tcp.wave.wave24z.com

on the command line. If everything is fine, you should see something similar to:

You might want to make sure that you can ping wavesandbox.com from your host. Should you try to

dig +short -t SRV _xmpp-server._tcp.wavesandbox.com

and don’t get a result, don’t worry, this is intended by design! The process is as follows:

FedOne initiates disco request for wavesandbox.com

Openfire tries to look up the SRV record for _xmpp-server._tcp.wavesandbox.com and fails, as we do when trying to dig it

Openfire falls back to the A record for wavesandbox.com, which succeeds, but the server there listening on the standard port (5269) fails the disco request, thus printing a <remote-server-not-found xmlns=”urn:ietf:params:xml:ns:xmpp-stanzas”/> to the console

FedOne falls back to the JabberID wave.wavesandbox.com and initiates another disco request

Openfire looks up the SRV record for _xmpp-server._tcp.wave.wavesandbox.com and should succeed, thus initiating the connection

XMPP server dialback negotiation starts (this will soon be TLS) and the connection should succeed

FedOne starts talking to wave.wavesandbox.com

(Thanks to Google’s Jochen Bekmann for clarifying this part.)

So running

dig +short -t SRV _xmpp-server._tcp.wave.wavesandbox.com

on your server should return a similar result to the one I’m getting on mine:

Once you’ve mastered DNS configuration, the next step is to create certificates and modify FedOne’s configuration files.

Part III: Welcome in certificate hell

Next to the correct DNS setup, certificates seem to be the second most important part of getting federation to work. Shortly after the announcement to open up the sandbox environment, the instructions given on the certificate wiki page have not been all that clear and the wave-protocol list suggest that many of us had severe problems getting the correct certificate setup done.

It has been revised since and now is way better and more precise. While I encourage you to first read through the instructions there, I highlight some of my findings and hope it helps.

Besides the sandbox environment Google has been operating two other environments for developers to test connections before: initech-corp.com and acmewave.com. You will not be able to ping initech-corp.com nor acmewave.com.

will reveal they do exist! The FedOne source code currently available in the repository comes with the initech-corp.com Wave server set up as a default ping target, so don’t worry when you see your FedOne server trying to contact initech-corp.com in your server logs. We’ll come to that later.

As to the certificates it is important to understand that

initech-corp.com accepts self-signed certificates,

acmewave.com only accepts CA-issued certificates,

wavesandbox.com only accepts CA-issued certificates

We are planning to federate with the sandbox environment, so next we are going to create CA-issued certificates. Here is how:

Change to your wave-protocol directory and create the make-csr.sh script:

StartSSL, the free service I’ve used to create my certificates does not accept 1024 bit long signing requests, so I’ve slightly changed the script provided by Google to generate a 2048 bit CSR.

Make the script executable:

chmod +x make-csr.sh

and run it as

./make-csr.sh wave24z.com

Now, leave everything blank (hitting Enter all the time) except the Common Name. For the common name provide wave.<yourdomain>. Make sure you’re entering the wave subdomain and not just your main domain!

Next head over to StartSSL. Their website looks and feels a bit 1980-ish and it did not work for me in Safari. I ended up using Firefox 3.5.4 which worked just fine.

On the StartSSL website

select Sign-up and sign up

StartSSL installs a certificate into your browser, so if you subsequently visit their website, you can log back in by clicking Authenticate

select Validation Wizard > Domain Name Validation

enter your domain name, for me this was wave24z

StartSSL is going to send a verification code to you via email in order to make sure, that you’re the owner of the domain. That’s why you do need to have access to emails sent to postmaster/hostmaster/webmaster or dns-admin at your domain. Select the appropriate option, hit Continue and wait for the verification code to show up in your inbox

paste the verification code into the text field and finish the process

Once you’re domain has been verified continue and create the certificate:

select Certificate Wizard

chose XMPP (Jabber) SSL/TLS CertificateDuring my first attempts I’ve chosen Web Server SSL/TLS Certificate and while I’m not exactly aware of the differences between the both options, only the XMPP certificate worked for me.

skip the Generate Private Key step by hitting Skip. You will instead use the Certificate Signing Request created before.

go back to your wave server console, print out the contents of the CSRcat wave24z.com.csr
and paste them into the text field at StartSSL

for the subdomain enter wave

wait until StartSSL sends you an email confirming that your certificate is ready

Go to Tool Box > Retrieve Certificate and grab your certificate

Next we’ve got to install the certificate for FedOne:

on the FedOne console create a new file wave24z.com.crt and paste the contents of the certificate into it

make sure that the key file (wave24z.com.key) exists. This has been created when we ran make-csr.sh.

Upload the file to the wave-protocol directory or create an empty file and paste the contents of the downloaded pem file into it

Once you’ve done all of these steps you should have the following files inside your wave-protocol directory:

wave24z.com.key

wave24z.com.crt

wave24z.com.csr

sub.class1.server.ca.pem

This is a perfect moment to make a backup of these files.

Though StartSSL has a Tool Box option that states Revocation Request you always almost receive an email stating that they are not going to revoke your certificate and thus not letting you create a new one. So in case you lose your key file and have to set up a new server, you’re doomed. Go ahead, and backup the CSR, your key file and your certificate now!

Next we’ve got to configure FedOne to use the certificates:

Make a copy of run-config.sh.example viacp run-config.sh.example run-config.sh

Open run-config.sh with your favorite editor, I’m using nanonano run-config.sh

Comment out “# echo “You must configure the run-config.sh script” ; exit 1″ by putting a # in front of it

Set WAVE_SERVER_DOMAIN_NAME to your domain

Set WAVE_SERVER_HOSTNAME to your domain

Set XMPP_SERVER_SECRET to the secret you’ve configured during Openfire setup

Set CERTIFICATE_FILENAME_LIST to CERTIFICATE_FILENAME_LIST=${WAVE_SERVER_DOMAIN_NAME}.crt,sub.class1.server.ca.pem
Make sure you changed “.cert” to “.crt“, you add “,sub.class1.server.ca.pem” file and there is no blank between the comma and the file name!

Set WAVESERVER_DISABLE_SIGNER_VERIFICATION to false

Please follow these instructions with much attention to detail, as they are super important. Here is a full copy of my run-config.sh with the shared secret blanked out for obvious reasons:

#!/bin/bash
# Configuration for the FedOne run scripts.
#
# Copy this file to run-config.sh and configure the variables there.
# Please see http://code.google.com/p/wave-protocol/wiki/Installation for
# instructions on how to configure the flags.
#
# Remove this line after configuring
# echo "You must configure the run-config.sh script" ; exit 1
### Variables common to the server, clients, and agents
###
# Domain name of the wave server
WAVE_SERVER_DOMAIN_NAME=wave24z.com
# Host name and port the wave server's client frontend listens on
WAVE_SERVER_HOSTNAME=wave24z.com
WAVE_SERVER_PORT=9876
# The version of FedOne, extracted from the build.properties file
FEDONE_VERSION=`grep ^fedone.version= build.properties | cut -f2 -d=`
### Server-specific variables
###
# These will probably need to be changed
XMPP_SERVER_SECRET=***********
PRIVATE_KEY_FILENAME=${WAVE_SERVER_DOMAIN_NAME}.key
CERTIFICATE_FILENAME_LIST=${WAVE_SERVER_DOMAIN_NAME}.crt,sub.class1.server.ca.pem
# These should be okay to leave alone
CERTIFICATE_DOMAIN_NAME=$WAVE_SERVER_DOMAIN_NAME
XMPP_SERVER_HOSTNAME=$WAVE_SERVER_DOMAIN_NAME
XMPP_SERVER_PORT=5275
# Set XMPP_SERVER_IP to localhost if the XMPP and FedOne servers are
# running on the same host
XMPP_SERVER_IP=$XMPP_SERVER_HOSTNAME
# Set true to disable the verification of signed deltas
WAVESERVER_DISABLE_VERIFICATION=false
# Set true to disable the verification of signers (certificates)
WAVESERVER_DISABLE_SIGNER_VERIFICATION=false

In a nutshell this will make FedOne look for the wave24z.com.key, wave24z.com.crt and sub.class1.server.ca.pem files.

The final step is to install the StartSSL intermediate certificate into the Java keystore. Do so by submitting:

Part V: Running the console client and waving along

Fire up the console client./run-client-console.sh <username>
providing the username you’ve configured in Openfire. For me this is ./run-client-console.sh ralf.

Create a new Wave and open it/new
/open 0

Add a sandbox participant/add <name>@wavesandbox.comFor me this would be /add 24z@wavesandbox.com

Type in your messages and have fun waving along!

That’s it! We’re done! Your very own Wave server is federating with Google’s sandbox environment. We’ve come a long way!

Final remarks and hints

Openfire Session monitor

Openfire’s web based admin tool provides a convenient way to monitor component and server sessions. If federation is successful, you should see something similar to this in Sessions > Server Sessions:

If you click on one of the sessions Openfire indicates packets sent and received. You should see those numbers increase when exchanging blips with your federation partner.

The ping server setting in run-server.sh

The value set for –xmpp_server_ping in run-server.sh has no impact on the servers you can potentially federate with. It’s really just meant to configure which server FedOne should automatically start to ping after startup. No matter what you’ve configured there, it’s the /add command submitted via the console client which invokes the connection to whatever your federation partner should be.

So it’s perfectly fine to have the default ping server (–xmpp_server_ping=”initech-corp.com) configured in run-server.sh and add participants from wavesandbox.com and other FedOne servers.

CAcert certificates

I have not been successful using certificates issued by CAcert. Discussions on the wave-protocol Google Group suggest that it is doable, but I gave up trying and went over to StartSSL. According to posts from Google’s Dirk Balfanz on the official wave-protocol list suggest, that CAcert.org issued certificates should not be expected to work, as wavesandbox.com only understands StartSSL certificates for the moment.

CAcert would be the better option as they allow you to revoke at any time and create new certificates. In case you’ve been successful, please let us know in the comments.

Latency

While federating with the sandbox environment I’ve experienced quite some latency. As Google points out federation is still in a very early stage. Thinks will remain a bit shaky for the time being. Once your server logs indicate that you’ve successfully connected to wavesandbox.com and you start typing on both ends, you might not see the result immediately.

Stay patient. I’ve experienced times where it took some 5-10 seconds until blips showed up on the other end.

You’ll also occasionally run into this funny state when things get out of sync:

Testing

If you’d like to join the discussion, we’ve create a public Wave over at the consumer preview environment (you need to be a Wave member @googlewave.com to join).

Do a search for

with:public “Google Wave Federation Testing”

to find it and feel free to join!

Finally, here are contact options, should you require a federation partner:

Sandbox environment: 24z@wavesandbox.com

Preview environment: ralf.rottmann@googlewave.com

My own FedOne environment: ralf@wave24z.com

I’ve also created logins to the server running at wave24z.com which allow you to use the command line client to test federation:

ssh public@wave24z.com

The password is wavepower and public2, public3, public4 and public5 should work, too. As wave24z.com is my personal playground, the server might be down occasionally.

Last but not least, you might want to follow me on twitter, as I’m continuing to publish my progress and findings there:

Final words

Well, I recently co-founded GrandCentrix, a company that offers a cloud platform that makes it easy to monetize massively scaling mobile services. Part of the GrandCentrix platform is to dramatically simplify integration into third party services, like carrier payment gateways, social network APIs and geo location services.

With major corporations starting to announce their Google Wave support, we believe it’ll ultimately bring a competitive advantage for our customers, if they can easily incorporate Wave as a communication channel for their mobile services.

That’s why. :-)

Thanks for staying with me for such a loooooong blog post and don’t forget to share your experience with Google Wave federation in the comments!

Thanks for the instructions, I’m almost there. I.e. I get receipts from wavesandbox, but I’m not seeing anything in the wave sandbox client. Do you think it’s a certificate issue? I’m using StartSSL certificates…