Archive

This is one of the last interviews I made to the speakers of Erlang Factory London 2011. One of my favourite erlang developers is Michal Ptaszek. Michal works at Erlang Solutions Ltd in Krakow. He is pretty famous among ejabberd developers and XMPP lovers.

Ask and Aswer

Paolo – Michal, can you introduce yourself to our readers? Michal – Hello, I am Michal Ptaszek – Erlang passionate and evangelist, someone who became seduced by the beauty of logic of maths and algorithms in his early childhood. Currently I work for Erlang Solutions, trying to cope with the complexity of massive scale Internet Messaging systems. From personal perspective, I have done my masters in Krakow University of Science and Technology in Grid systems field (Erlang obviously!), currently I live in Krakow, Poland, love jazz and modern literature (especially latin). Some day would like to find some time to learn how to play sax.

Paolo –How did you “discover” Erlang? Michal – It has started in the early 2008, when Joe Armstrong came to Poland to give a talk on Erlang concepts for Student’s IT Festival. Of course almost no one has ever heard of the language (or. even if, never touched it), and as Joe is such a great speaker, he planted some seeds of curiosity in my mind.

Paolo – How long did it take to you to master Erlang? Michal – I wouldn’t say I’ve mastered Erlang. Comparing to such people as Ulf Wiger or Robert Virding, I’m still in my infancy 🙂 The great thing in Erlang is that you do not have tons of code everywhere: language is simple, standard libraries are manageable, even VM: if you clench your fists you can read and understand the entire code in few weeks. In my opinion people are in never-ending continuos process of developing themselves: when I look at my code written a year/two ago I feel ashamed of myself and would love to throw it away and rewrite it in some nicer way. I guess at the time when I discover there is nothing to improve I will be forced to change the profession (or at least the programming language) 😉

Paolo – How did you find your first Erlang related job? Michal – Few months later after Joe’s visit to Poland, Erlang Solutions (Erlang Training and Consulting back then) decided to open an office in Krakow: I did not hesitate a lot and applied for the internships. The office grew from 3 to 12 people since then and is extending all the time: I guess it was a really good choice.

Paolo – What are the things you love most of Erlang? Michal – Just to name two most important things: – built-in parallelism and distribution – I can not even imagine how do people distribute their applications and protect their critical sections using semaphores and mutexes. It’s so 1990… – fault tolerance – one of the least appreciated things in the language: ability to isolate the errors, propagate them in the controlled way and recover automatically. In the systems that run on top of tens/hundreds of machines crash of one of them is not a question of ‘if’ but ‘when’.

Paolo – During Erlang Factory, you will talk about Erlang and XMPP: can you give to our readers a brief description of your course? Michal – Yes, together with Michal Slaski we are going to teach a one day course on Erlang and XMPP (ejabberd in particular). During those few hours we would like to show how the things work under ejabberd’s hood, why it is not a best idea to keep the default configuration, what are the low hanging fruits for ejabberd, BEAM and OS enhancements, how can we monitor existing server, how to cluster several daemons together in one command and finally: what are the ways to extend the functionality programmatically.

Paolo – Who should follow your course and why? Michal – Definitely people who are interested in Internet Messaging, even not necessarily in XMPP. Architecture of message/presence router, as well as the easiness of extending current functionality, federating with other protocols or finally scalability might be stimulative to someone who plans to set up a startup or replace his current messaging server with something better.

Paolo – Why should an XMPP programmer use Erlang as development language? Michal – Mostly because of the concurrency and scalability. Erlang actor process model fits perfectly in the concept of independent user sessions and communicating with each other using asynchronous message passing.

Paolo – In your opinion, what is the most important thing to keep in mind when developing with Erlang and XMPP? Michal – Global state/shared data is a root of all evil. The less things you share – the better you can scale. XMPP systems have tendencies to grow rapidly: it’s basically the linear relation to the number of simultaneously connected users. Shared global state will not pop up as a bottleneck until it is too late: when users are storming our gates and we realize that things should have been done in a different way long time ago. And there is also one more rule, that people often forget about: KISS. IM systems are really simple, let’s do not overengineer them

Paolo – Can you recommend to our readers some books, sites, or blogs that will help them in the world of Erlang and XMPP? Michal – Definitely take a look at Jack Moffitt’s both book (“Professional XMPP Programming”) and webiste(http://metajack.im/). Another worth-reading blog is Stefan Strigler’s http://blog.jwchat.org/. To be honest: there is still no book on Erlang and XMPP in particular. Maybe it is a good time to change it? 🙂

In the previous post we discussed ow to enable anonymous authentication in ejabberd in order to use Tsung for our testing purposes.

Tsung (a.k.a. Tsunami) is a shell testing tool fairly easy to use when dealing with XMPP, it is mainly used to simulate thousands of users for testing the scalability and performances of IP based client/server applications.

For XMPP test purposes it allows:

Authentication (plain-text, digest and sip-digest)

presence and register messages

Chat messages to online or offline users

MUC: join room, send message in room, change nickname

Roster set and get requests

Global users' synchronization set on specific actions

raw XML messages

PubSub

Multiple vhost instances support

First thing first: install Tsung! Download the source from http://tsung.erlang-projects.org and just follow the common procedure configure, make, make install (you may need sudo).

When you start Tsung (we will see how) it executes a set of operation based on an xml file (tsung.xml) that you easily configure by using a text editor. This file is inside the directory ~/.tsung but sometimes it is not present so I suggest you to copy one of the examples you can find in the directory “examples” of the installation package, copy it there and rename it.

The Great Wave Of Kanagawa

Let’s take a look at tsung.xml.

Scenarios start with clients (Tsung cluster) and server definitions, so for a non-distributed load, you can use a basic setup like:

You can put more than one client or one server in you tsung.xml, using either a string or an ip-addres to identificate them. In case of multiple servers, a round robin algorithm will be used to chose a server.

Let’s define now a load progression by specifing a set of arrival phases to the server:

The phases will happen one after the other and the test will finish when all users have ended their session. Notice that duration of the test can be much longer than the duration of arrival phases, therefore if you want to stop the test after a threshold duration (even if phases are not finished or if some sessions are still active), you can do by setting a duration attribute to load:

In many previous posts, I gave you some ideas on how to build an external component for XMPP servers; now it is time to test your work!

Tsung is a stress testing tool written in Erlang that you can use to test HTTP, WebDAV, LDAP, MySQL, PostgreSQL, SOAP and XMPP servers; it can simulate hundreds of simultaneous users on a single system in a single server or in a clustered environment therefore Tsung is the right tool to test our XMPP services.

First of all: when you use Tsung, you simulate few or many users….the problem is that you need to authenticate these users against the server, so you have two solutions: the first one is to create all the users…the second one (the smartest) is to enable anonymous authentication in your XMPP server.

Anonymous authentication is a standard login using classical login and password mechanisms, but where password is accepted or preconfigured for all anonymous users. This login is compliant with sasl authentication, password and digest non-sasl authentication. For more information see this.

On ejabberd, to provide anonymous authenticatio to users you have to tune your configuration file as follows:

1) set up two virtual hosts on the server, one public (any user can login) and one private (login only with valid credentials):

{hosts, ["private.yourdomain", "public.yourdomain"]}.

2) set the global authentication method that you will use. In our case we will authenticate against ejabberd internal database, but alternative solutions can be MySQL, PostgreSQL, LDAP and so on:

{auth_method, internal}.

3) At the end override the host_config options to provide anonymous login to public host’s clients:

Just start/restart your ejabberd from this point you can connect to the virtual host “public.yourdomain” with any pair (JID, pwd) but you will be able to authenticate against “private.yourdomain” only with a valid pair.

Ok, now you have anonymous authentication enabled in you XMPP server!

This week I’m really busy 😦 , so I will teach you how to use Tsung and how to configure it for our purposes in the next post.

A couple of posts ago, I introduced EXMPP, the Erlang library developed by Process One for XMPP.

In that post we used EXMPP to connect an external component to eJabberd server by using the XEP-0114.

This time I will show you how to process streams directed to our component, analyze them and reply to the sender.
First of all I used a gen_server behaviour and put all the code regarding the handshake inside the init([]) method:

Where get_conf_value(Value) is simply a function I implemented to parse a configuration file.

EXMPP provides to developers a set of records (a record may be thought as a complex data structure and I will write about
them in the future); among these records we need in our code the one representing a received_packet.
It is defined as following:

After the handshake we can send any kind of stanzas defined in XMPP realm towards the server we are connected to as long as
we have a reference to the session we established before.

A really cool feature of EXMPP is that two different processes spawend by our gen_server can send a stanza at the same time over the same session and the stanzas will not be mixed during their “trip” to the XMPP server.

About incoming stanzas, you must know that all of them will be processed by the gen_server as normal Erlang messages.

I told you before that all the stanzas can be handled by processes spawned by the gen_server, so it seems sensible to spawn
a new process any time a packet is received; this new process will elaborate the stanza and if required will send a reply stanza, then it
terminates.

In practice whenever we receive a Packet we spawn (without linking!) a new process which takes as input the current session and the packet itself.
This process is represented by the following function:

Here I am again. Sorry for the time it took me to write a new blog entry, but I was preparing the speech for my bachelor thesis..so as you could understand that took all of my free time.

My thesis explained how social networking could be integrated in an XMPP environment, and I have to say that during the time it took me to design and write the code I became an XMPP nuts. XMPP is a protocol aimed to real-time messaging and presence, but it allows developers to use many extensions to implement different kind of services.

For my thesis I used as XMPP server the software Tigase and since it is developed in Java, I wrote all of my code in the same language, but I was aware of the fact that a XMPP server developed in Erlang and called eJabberd exists so I decided to port all of my code in Erlang.

In this post I will teach you how enable an external component in eJabberd and how to code it using Erlang and EXMPP a library provided by Process One. First of all we have to enable the external component in the eJabberd configuration file; by default you can find it in /etc/ejabberd, so:

If you don’t know about XMPP and external component I can just tell you that the names I specified in the list after hosts represent the names that will be used to identify this component in the XMPP network, while the value of password is just the password used by the component to perform the handshake with the server. For more info you can read this.

At this point save the configuration file and restart eJabberd.

Now let’s use EXMPP to connect from the Erlang shell to eJabberd as if we were one of the two components.

In the code above, I first started the exmpp application, then I created a Session for the XMPP component and made the authorization stream by passing to the function exmpp_component the session, the name used by the component and the password needed for the handshake.

Then I connected it to the eJabberd server by using the function connect, which needs the session, the name of the server and the port on which it is listening to.

After that just connect the component by using the function handshake.

Now, you can log in to the server with one test account of yours and perform a service discovery, the result should be something like:

PSI: Service Discovery

Ok, I know this post was kinda short, but I promise that next one will be much better, and in it I will teach you how to receive and send XMPP stanzas from our component!