O'Reilly Media books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://safari.oreilly.com). For more information contact our corporate/institutional sales department: 800-998-9938 or [1].

Nutshell Handbook, the Nutshell Handbook logo, and the O'Reilly logo are registered trademarks of O'Reilly & Associates, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O'Reilly & Associates, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. The association between the image of a bare-throated bellbird and the topic of Jabber is a trademark of O'Reilly & Associates, Inc.

LEGO� is a registered trademark of The LEGO� Group. MINDSTORMS and Robotics Invention System are trademarks of The LEGO� Group. All other trademarks, service marks, and the like are the property of their owners.

While every precaution has been taken in the preparation of this book, the publisher and the author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein.

Preface

What is Jabber? Depending on who you ask, the answer to that question could be any one (or all) of the following:

Jabber is a technology.

Jabber is a protocol (or set of protocols).

Jabber is an XML-based Instant Messaging (IM) system.

Jabber is an implementation of the set of protocols.

Jabber is an idea whose time has come.

In fact, all these answers are right. Jabber is a set of protocols expressed within XML that allow people and applications to converse with one another. Sure, TCP sockets, STDIN/STDOUT, infrared, voice input, and teletype mechanisms all allow people and applications to converse; the difference is that Jabber provides a structured, extensible framework for exchanging all kinds of information.
This is all rather abstract, so what do we mean by "extensible framework for exchanging information"? Taking it one word at a time:

Extensible

Jabber's substrate is XML. XML is inherently extensible in the sense

that tags can be added in a hierarchical sequence. Namespaces in XML

allow us to keep track of the meaning and organization of these tags.

Framework

You can put together a system for exchanging information using many

different tools. The point about the "framework" is that the

information exchanged and the entities that are exchanging it are

contextualized, bringing meaning and structure to the interactions.

Exchanging

Conversation is two-way, and it takes many forms

question and answer, notification, compartmentalized discussion,

and simple chat. Jabber supports all these different types of conversation and more.

Information

One doesn't really say that information is exchanged in a

conversation, but when you bring applications into the mix of

conversing entities, it may well be the case. And it's not just

information exchanged in the form of conversations but also

information about the entities themselves that flows across this

context framework. It goes without saying: Jabber is an instant

messaging (IM) system.

Instant messaging was Jabber's original raison d'�tre. Many deployments of Jabber software are to provide IM services, but Jabber is more than IM�certainly more than the phrase "instant messaging" represents. In this book you'll find out why this is so and how you can deploy solutions with Jabber that are more than mere chat. But most importantly, Jabber is fun!

Like chess, which has a small set of rules but countless game possibilities, the technologies employed in Jabber and the Jabber protocol itself are straighforward. The possibilities are almost limitless. Furthermore, because of a fundamental design feature[1] (you might call it a "philosophical" angle), implementing Jabber-based solutions to your problems can be fun�really!

The History of Jabber

Jeremie Miller started the Jabber project early in 1998, and it was announced to the public in January 1999. To understand why Jabber came about, and in the form it took, let's look briefly at what existed in the IM world before Jabber.

The Pre-Jabber History

While the concept of live chat systems has existed for many years, with commands such as Unix's talk and VMS's phone, IM as we know it today existed as a concept and a handful of systems from companies such as Mirabilis, AOL, Microsoft, and Yahoo!. These systems (ICQ from Mirabilis, AIM from AOL, MSN from Microsoft, and Yahoo!IM from Yahoo!) allowed their users to chat to one another and avail themselves of IM-related services. However, an AIM user couldn't chat with an ICQ user, and MSN users couldn't interact with Yahoo!IM users. Each system was effectively closed to the outside world.

Furthermore, the protocols that these systems used were also mostly closed�proprietary�which meant it was difficult to find clients for these IM systems other than the ones supplied by the IM system owner.

Finally, the systems themselves were monolithic: multiple clients but a single server (or server farm). Although the companies were able to invest time and money into the problem, the fact remained that a monolithic architecture presented a scaling problem. Perhaps more relevant than that, companies that wanted to use IM services internally had to accept the fact that the conversations would be carried through the systems of a third party�namely the owners of these public IM systems. This was no more desirable than for a company to run its internal email using a public email service such as Hotmail.

Of course, these systems did have their advantages. The clients were accomplished and easy to learn and use, and as long as your correspondents were using the same IM system and you didn't mind your messages being carried by another organization (for private individuals these wouldn't be unique circumstances; again, we are led to the email services parallel), then you could leave the system management to someone else and get on with chatting.

Scratching an Itch

Having all your contacts use the same IM system is all well and good in theory but in practice is rarely the case. (If you're like me and have only a few friends, then this is not so much of a problem.) Jeremie Miller had correspondents in different IM systems and consequently had to have different IM clients running on his desktop to keep up with them all. Many great software projects stem from a personal "itch" that someone wanted to scratch. This was the primary itch that Jeremie had�a single client for all IM interaction: panacea.
Of course, one obvious solution would be to build a single client that supported all of the IM system protocols, but this approach had two drawbacks:

The proprietary nature of the protocols made it harder to implement the support required and would make the client overly complicated.

Every time the protocol, which wasn't under his control, changed or a new one came along, the client would have to be modified�a task not practical for a large user base. On top of that, GUI programming isn't everyone's cup of tea, and Jeremie preferred a solution that allowed him to concentrate on the underlying problems at hand and let others build the GUIs.

Along Came Jabber

So Jeremie resolved to create a solution that had the following characteristics:

It would have its own internal protocol, based upon XML. This protocol should be:

Simple to understand and implement

Easy to extend

Open

The complexity of bridging the disparate proprietary IM protocols would remain at the server, each bridge being a plug-in module.

All the clients would have to implement only the single, simple open protocol; everything else would be implemented at the server. He called this solution "Jabber." The main architectural feature for Jabber that Jeremie strived for was that it should be simple enough for anyone to implement a Jabber server of his own. Unlike a centralized server environment, with all of the traffic routed through a central point, Jeremie envisioned that Jabber would be decentralized, allowing individuals, companies, and public organizations to run their own servers. This is particularly relevant for internal-only, IM-style corporate communications. Just as email servers are used to exchange mail using the Simple Mail Transport Protocol (SMTP), the Jabber servers are able to connect and exchange IM traffic whenever necessary. Figure P-1 illustrates Jabber's distributed architecture, with two separate Jabber servers serving separate users.

Being open meant that Jabber could benefit from the help of anyone who wished to lend a hand, and administrators were empowered to be able to find and fix problems themselves if they so wished. ß Being XML-based, as opposed to another binary format, for example, meant that the protocol streams were easy for humans to read, extensible, and readily integrated (a great range of XML parsing and construction tools is already available).

Being distributed meant that the Jabber system would belong to the people and that some of the scalability problems would be avoided. There remain some scalability issues, of course. Client-server communication that is TCP socket-based suffers from limitations of this technology. There are, however, initiatives to overcome these limitations with multiplexing techniques such as jpolld and dpsm (see http://download.jabber.org).

All of these features made for a good IM system design. But why stop at IM? Consider the client as an implementation of a simple protocol to exchange messages and presence information in XML structures and use plug-in services at the server, and what do you have? A language- and platform-agnostic XML routing framework.

Good grief, what a mouthful! This is why my response to "What is Jabber?" is usually just:

A really great technology!

IM System Features

This book assumes you have a basic knowledge of features commonly found in IM systems. In case you don't, here's a brief rundown of features relevant to what we'll be covering:

Presence

In many cases, there's not much point in sending a quick message to

someone if they're not there. Presence is a term used in IM to

describe the technique of exchanging information, in a controlled

manner, about availability (or unavailability).

The idea is that

when you connect to your IM server, your client sends an "I'm here"

message that is relayed to your correspondents. It does the opposite

when you disconnect. During the time you're connected, you can vary

the information about your availability to reflect your immediate

situation ("just popped out for coffee," "working on my resum��don't

disturb me!").

Buddy List�/roster

Both terms (the former comes from the original IM systems, the latter

from Jabber) refer to a list of correspondents with whom you regularly

communicate and from whom you receive presence information. Depending

on the IM system, the list may be stored on the client or on the

server. Storing the list on the client has the (tenuous) advantage of

being accessible when you're not connected to the server. Storing the

list on the server means that you have a consistent roster content

regardless of the client or workstation you happen to use�the list

travels with you.

Jabber stores the roster on the server.

Push and pull

When you connect to an IM system, there may be information the client

needs to retrieve�pull�from the server (the roster, for instance).

This is under direct control of your client as it decides when to make

the retrieval. During the course of the connection, you'll receive

messages from your correspondents. You don't request these messages by

making a retrieve call to the server, as you would with the Post

Office Protocol (POP) or Internet Message Access Protocol (IMAP) to

retrieve email messages from the mail server; they're pushed to

you as they occur. : In other words, you could say that the client

must implement an event-based system, to listen out for and

subsequently handle the incoming information, by displaying a pop-up

window containing the chat message, for example. : The push/pull

system lends itself well to traffic other than IM traffic.

Client-server

It almost goes without saying that, like other IM systems, Jabber has

a client-server architecture. Clients and client libraries that

implement the Jabber protocol, such as Net::Jabber,

Jabber::Connection, Jabberpy, and

JabberBeans, are available for many languages (here, for

Perl, Python, and Java).

With Jabber, it is especially relevant to

stress that the "weight balance" in complexity terms, between the

client and the server, comes down heavily on the server side. Not only

does this mean that the complexity remains where it should be�on the

server�but also makes the task of writing clients easier and the

resulting software lighter. ; Multiple versus single server

We've already mentioned that the Jabber architecture does not dictate

a single, centralized server. Not only does this mean that

organizations can implement their own private system, but also that

developers are free to install their own server and develop new

plug-in services in addition to the IM bridges already available.

What's Inside

This book is not particularly about IM per se. Nor is it about the bridges to other IM systems. It's about the essence, the ideas, and the potential behind that concept and reality called Jabber.
You will learn about the Jabber protocol and how to use Jabber's technology to implement not only IM-based solutions but also solutions that don't involve inane chat. You'll learn how to install and configure your own Jabber server. You will discover more about the features of Jabber that give it its propensity for being an ideal messaging glue for many communication solutions; all of Jabber's technology features�the building blocks and the protocol itself�are explained; and you'll get to know how Jabber can be implemented in a variety of situations�some involving IM, others not�through a series of application and problem scenarios with fully working code examples, or recipes, in Perl, Python, and Java.

Here's a brief overview of what's in the book:

Part I

The first part of this book provides you with an introduction to

Jabber; you'll learn about its features, why it's more than an IM

system, and how to install and configure a Jabber server of your own.

Chapter 1

We begin with an imaginary conversation with human and application

participants that shows how Jabber provides the supporting messaging

"plasma." A short script shows how simple it is to make use of

Jabber's power.

Chapter 2

We take a look at some of the features�the nature�of Jabber, to

understand why Jabber is more than just an IM system. The features

introduced in this chapter will be revisited as core building material

for our recipes in Part I.

Chapter 3

Here you'll learn how to retrieve and install the Jabber server and

perform minimal configuration, enough to be able to fire it up and use

it as a basis for the recipes in Part I. Some troubleshooting and

monitoring tips are also included.

Chapter 4

Once we have our Jabber server installed and running, we take a closer

This chapter looks at the steps needed to create and authenticate with

a user and examines the different types of authentication. We also

build a utility to create users in order to generate users for our

recipes.

Chapter 8

This chapter looks at some simple examples of Jabber deployment using

basic features of message and presence, including presence

subscription.

Chapter 9

Extending the themes introduced in Chapter 8, we look at how messages

can be extended to carry custom and compartmentalized data and how to

write a 'bot that serves in a conference room. Furthermore, we look at

how components are defined and written and examine how different event

models can coexist with Jabber's.

Chapter 10

This chapter shows diverse applications of Jabber, from building an

online address book using Jabber as infrastructure, through exchanging

XML-RPC-encoded requests and responses over Jabber, to extending the

client scope of SAP's R/3 business software.

Software Used in This Book

The recipes in this book come in varying flavors, some in Perl, some in Python, and some in Java. These examples�to a greater or lesser degree�make use of prewritten libraries that provide at least the basic services needed to connect to a Jabber server and exchange data with it. Here's a summary of the versions of the languages used in this book, along with those libraries that are used, what features they offer, and where they're available. In addition, references to all of these libraries can be found on the Jabber development web site, http://dev.jabber.org. The installation instructions for the libraries can be found in the library packages themselves.

Java

The Java recipes in this book are written in Java 2 (J2SE�the Java 2 Standard Edition), specifically with the 1.3.1 version of the Java Development Kit (JDK).

JabberBeans is the name of the Java library for Jabber used in this book. It offers comprehensive coverage of the features needed to write programs that interact with Jabber servers: connection, authentication, and the management of Jabber elements passed between your script and the Jabber server.

The JabberBeans library can be obtained from http://jabberbeans.org. The version used in this book is 0.9.0-pre4.

Perl

The recipes have been built and tested with Perl 5.6.0, although earlier and later versions of release 5 will probably work just fine.

Two libraries are available for programming Jabber solutions in Perl. Both come in the form of installable modules and are of the object-oriented persuasion.

in addition to a host of higher-level features for manipulating all of

the Jabber elements and making use of standard and custom namespaces.

It relies upon a companion module XML::Stream, also

available on CPAN, that provides the underlying mechanisms for

creating connections to a Jabber server, as well as sending,

receiving, and interpreting (parsing) the fragments of conversation

between your script and that Jabber server.

The version of

Net::Jabber used in this book is 1.0022. The corresponding

version of XML::Stream used is 1.12.

Jabber::Connection

The Jabber::Connection module is available on CPAN and

provides the same basic features Net::Jabber does, albeit in

a more "RISC" (Reduced Instruction Set Computing) way. While it

provides similar functionality for connecting to and exchanging data

with a Jabber server, it offers, via a companion modulecalled

Jabber::NodeFactory, a lower-level API�similar to that in the

Jabber server itself�for constructing and manipulating the Jabber

elements. There are no high-level features; instead, you build your

own using the building blocks that the module provides. : The version

of Jabber::Connection used in this book is 0.02.

Python

The Python examples have been written with Python 2.0.

Jabberpy is the name of the Python Jabber library used in the Python recipes in this book. As with Perl's Net::Jabber library set, Jabberpy provides its feature set from two separate libraries�jabber, which provides connectivity, authorization, and callback functions such as Net::Jabber and Jabber::Connection, and xmlstream, which provides the basic connectivity and parsing functions such as Net::Jabber's companion XML::Stream.

Finally, there's a Jabber conference room called jdev where many of the Jabber developers hang out. Just point your Jabber client at conference.jabber.org and drop by for a chat.

Conventions Used in This Book

The following typographical conventions are used in this book:

Bold

Used for commands, programs, and options. All terms shown in bold are

typed literally. ; Italic

Used to show arguments and variables that should be replaced with

user-supplied values. Italic is also used to indicate new terms, URLs,

filenames, file extensions, directories, and to highlight comments in

examples.

Constant Width

Used to show the contents of files or the output from commands.

Constant Width Bold

Used in examples and tables to show commands or other text that should

be typed literally by the user.

Constant Width Italic

Used in examples and tables to show text that should be replaced with

user-supplied values.

Note:

These signify a tip, suggestion, or general note.

Warning:

These indicate a warning or caution.

How to Contact Us

We have tested and verified the information in this book to the best of our ability, but you may find that features have changed (or even that we have made mistakes!). Please let us know about any errors you find, as well as your suggestions for future editions, by writing:

O'Reilly Media, Inc. 1005 Gravenstein Highway North : Sebastopol, CA 95472 (800) 998-9938 (in the U.S. or Canada) (707) : 829-0515 (international/local) (707) 829-0104 (fax) You can also send us messages electronically. To be put on the mailing list or to request a catalog, send email to:

[3] We have a web site for the book, where we'll list examples, errata, and any plans for future editions. The site also includes a link to a forum where you can discuss the book with the author and other readers. You can access this site at:

Acknowledgments

To my dearest wife, Sabine, who patiently and quietly took on all the demands of daily life for us during this project, allowing me to devote myself to researching and writing. To my son, Joseph, who knew I was writing a book but also knew that helping him build train tracks, drawing, and going on bicycle rides was always good tonic for a mind congested with XML.

To my grandpa, Harold Lomax, who never tired of reading through what I had written to check and correct grammar and punctuation.

To my editor, Chuck Toporek, who not only has great taste in music, but also has a sense of humor to match his infinite patience. Chuck taught me a great deal about writing and thinking, without me even realizing it was happening. Also, many thanks to David Chu, Chuck's editorial assistant. It's easy to miss the people behind the scenes when they do such a good job, so thanks for pushing my SGML files through when they needed to be checked by Tools, for coordinating the technical review, and for assisting Chuck to make this book happen. Thanks also to Lenny Muellner, my SGML guru. He helped me with my SGML queries and always kept me on the right track.

To the reviewers of this book, Jens Alfke, Matthew Allum, Michael Bauer, Piers Harding, Jeremie Miller, and Thomas Muldowney, who gave up a lot of their time to provide me with wonderful feedback and great insights. Special thanks go to Piers, the best technical debating partner you could wish for; trying to keep up with him mentally is what drives me on.

And, of course, where would this book be without the very thing that captured my imagination and held it all this time? Thanks to Jeremie and the team of core Jabber developers for building something so fascinating. Thanks also to the Jabber developer community at large, a more friendly bunch of diverse people one could hardly hope to meet.

Finally, thanks to Tim O'Reilly and all of O'Reilly & Associates, Inc., for seeing Jabber for what it really is and for having confidence in me to write about it.