Friday, 18 December 2015

"I’ve been following TADHack and its related events for some time, and finally this month I got the opportunity to attend TADHack-mini Paris. Participants can join from remote too, but the personal full immersion is something different (even, ironically, when the topic is Real Time Communications, and more in particular WebRTC and Telecom APIs.

We met in central Paris [...] "

This is the beginning of the behind-the-scenes story about my TADHack participation. You can read my full article here.

Wednesday, 16 December 2015

I was very happy to see the news of the release of a new Kamailio module, authored by Victor Sveva.CFGT can be used to test call scenarios and see what routing logic was triggered in Kamailio.
Test calls need to be marked with a specific, configurable Call-ID pattern ('callid_prefix').
A JSON report is generated, with the possibility to choose what variables to dump into it.
This is going to greatly simplify testing, while potentially keep the logging to a minimum. Highly recommended.

Thursday, 15 October 2015

There are scenarios where TLS 1.2 is not just enabled, but the only one accepted.In these cases many clients fail to connect over HTTPS.I needed to be able to use 'git clone https://...' on CentOS 7, and since it was failing and I spent some time on a work around, I'm sharing it here.The system is a CentOS 7 host on DigitalOcean, with kernel

To be honest, I don't think I need to sale this very hard, so I'll proceed to a practical consequence: unit testing for puppet modules. Unsurprisingly, there's a app tool for that: rspec-puppet.
At least this is what I've been using for some time and find very useful and easy to use. I've even created some Jenkins jobs just to unit test Puppet modules.

You can find a tutorial for rspec-puppet here. Feel free to leave this article, read the tutorial, experiment a little and come back later.

What I wanted to share is some tricks/settings that I had to use, which I haven't found in one single place so far.

As you can see in the tutorial, rspec-puppet generates a dir skeleton for you (with the command 'rspec-puppet init'), to be populated with the tests for your module, then you just need to run 'rake spec' and have the unit tests run.

What I noticed though was that 'rake spec' didn't quite work, or didn't work as expected, and eventually I ended up with installing these dependencies (alas, with reference to a CentOS 7 host):

package { [

'bundle',

'puppetlabs_spec_helper',

'puppet-lint',

'rake',

'rspec-puppet'

]:

ensure => present,

provider => 'gem',

}

Then I found a better Rakefile (although I can't remember the origin, it must come from an official Puppet forge module. If you recognize it give me a shout and I'll give full credits):

The last important bit was the .fixtures.yml file, which allows to refer to 3rd party modules required by the module under testing.

Here's an example:

which basically says: "You can find mymodule in this directory, and please use stdlib from this other directory". In fact, for stdlib you should not use the local path (because it implies that stdlib is installed, and somehow defeats the point of unit testing) but the git URL. Since this installs stdlib from git at every run, I preferred using a host with it installed and refer to the local path instead. Not perfect, but handy.

Only then I could use 'rake spec' with satisfaction.

I hope you find this useful, and if you have any type of feedback please don't hesitate to add a comment.

Some of the readers may have noticed that I was using plain WebSockets ('ws://' URLs). It's recommended to use Secure WebSockets instead ('wss://' URLs), so I thought of playing with the 'ws' node.js module and "add TLS".

On github there's an example in this direction (see below), but I must admit I didn't understand some implications at first.

I thought the instantiation of an HTTPS server was just coincidental and meant to provide the web pages and scripts in the example, and that the configuration of 'ws' with 'ssl: true' and certificates was independent.

It turns out it's not. The best description of my understanding is that you need an HTTPS server to "decorate" the WebSocket module. The HTTPS server will take care of connection instantiation and encryption, while the WebSocket module, "listening" on the same port, will take over when the Upgrade request [1] from the client is received.

Here's a snippet of the solution I've adopted, based on the example above:

You can see that the version for plain WebSocket (commented out) had the configuration object passed to the WebSocket constructor (well, in fact, you just need to pass '{ port: 8080 }'), while the secure solution passes the entire HTTPS server object to the WebSocket constructor.

In this case we’re considering video-conference scenarios, possibly with many participants, and the ability to add some presentation features, like
highlighting the current speaker.

The architecture behind Jitsi VideoBridge aims to avoid a
centralized mixer (MCU), but at the same time prevent the complexities of a
Full Mesh approach.

Enter the SFU (Selective Forwarding Unit)
concept: the server component is “simply” a router of media streams among conference participants. A IETF Draft describes the behaviour expected for an SFU. Each participant receives one stream per each other user: it’s
then up to the receiving client to take care of stream presentation.

The SFU doesn't need to decode, mix, and then re-encode
all the streams, and for this reason it can use just a small amount of
resources, and scales well.

The critical point, as Emil is aware too, is dealing with mobile
devices: low available bandwidth and relatively limited computational resources make it
prohibitive for applications running on such devices to handle many different
streams, some of each potentially carrying high-quality video.

What can help in this case is the concept of “simulcast”:
compatible applications (like Chrome) are able to generate streams at different
quality levels (resolution and framerate) at the same time. There are still some difficulties in using simulcast, as witnessed in this post. Different topologies involved in WebRTC networks are also well described in this other article.

Anyway the receiver of such streams can then instruct the server
about which quality it is capable of receiving. Apps on mobile devices will get
the low-quality streams; browsers on desktops will require the high-quality
ones.

This is a clever solution, but I’d still like to experiment and see examples
of mobile apps managing in an acceptable way more than 2-3 incoming streams, regardless of the optimized bandwidth usage. And if you have any reference, please feel free to post a comment!

Tuesday, 3 February 2015

It's that time of the year when experts of Open Source software meet in frosty Brussels for two intense days of talks, conversations, and a good quantity of beer: FOSDEM.Being just 2 hours away from London with the Eurostar, the relevance/effort ratio is very high. Additionally, the event is free and held over the weekend, so it has a low impact on the normal job activity (although on the other hand it does have some on your family time, but you can't have everything, unless your kids are big enough to join you, which I'd recommend).Right after settling down in a lovely flat (AirBnB is a great choice) with three friends I started planning the sessions to follow. There are about twenty parallel sessions, so you must cherry pick. For day 1 I was oriented towards Configuration Management and Lightning Talks tracks. Day 2 had Virtualisation and Testing And Automation in my radar.As it turns out, FOSDEM is such a success that the rooms are filled incredibly fast and many have even long queues outside. It was definitely the case for the Configuration Management dev room, which I had to skip in favour of Infrastructure As A Service.So I got a basic understanding of Apache Mesos, for resource management of distributed systems, and GlusterFS, a distributed file system and general-purpose storage platform.Apache Mesos will come back to attention on Day 2, when I'll learn more about CoreOS Rocket for container orchestration (more on this on a following post).This year there wasn't any track dedicated to RTC (Real Time Communication), but nevertheless the topic was around a lot (not to mention the Internet Of Things track, which I gather has been very successful).

The Python devroom though offered an interesting cross between Python and WebRTC, with Saúl's "Python, WebRTC and you" (slides here). As the reader may know already, WebRTC doesn't mandate a signalling protocol, so this talk has been a good opportunity to show how conceptually easy is to interact with WebRTC's APIs (using rtcninja as wrapper), while at the same time managing signalling between browsers with a combination of Python3 and asyncio.This talk alone, from a teaching perspective, is very valuable. You can see:- The power of WebRTC: as long as the browser is compatible, you're just a few dozens of lines of code away from a new application.- You can use the web server you prefer: web content servicing and signalling are just tools around the RTC.- Unless you need some complex feature, you can leverage pure P2P and remove the server infrastructure from the equation.- WebSocket support is increasing [1], making the life of WebRTC architects easier.Of course things get more complicated for multiparty conferences, but there's something about this on a following post.[1] The popular node.js platform has an ad-hoc module for WebSocket support too. I wrote about it and its usage inside a docker container in a previous post.

Saturday, 24 January 2015

Docker is an incredibly useful tool to build prototypes of Linux hosts and applications.

You can easily build a network of servers inside a single virtual machine, with each server represented by a docker container. Clients can access the services on the same IP address, but different ports.

In this post I'd like to talk about a common prototype case in WebRTC platforms: a WebSocket server. This will be a node.js server and will run inside a Docker container (hosted by an Ubuntu Trusty VM).

The server logic can be as complex as you can imagine, but since it's not the point of this post I'll keep it as simple as the server example in the node.js websocket module:

The WebSocket server will listen on port 8080, accept incoming connections, send back "something" upon client connection, and log the content of the messages from the clients.

We can assume all the files in this article are in the same folder, and we're cd into it. The server logic is inside a 'server.js' file.

As explained in this interesting post from Ogi, you can find docker images with node.js all set and ready to be used, but the purpose of this post is to go a level deeper and build our own image.

Let's create a Dockerfile like this:
Even if you're not familiar with Dockerfiles, I'm sure you find this self-explanatory. The tricky bits are on line 13, where we symlink the nodejs executable to the desired '/usr/bin/node' (see here why), and line 16, where we install the node.js ws module via the npm package manager.

Line 18 tells docker what port this container is expected to receive connections to.

Line 20, the ENTRYPOINT definition, tells docker what command to execute when running.

(Remember that a docker container will run as long as there's a running command in foreground, and will exit otherwise.)

From inside the same folder, we can build our container image with:

docker build -t gvacca/nodejs_ws .

'gvacca' is my username, and 'nodejs_ws' is an arbitrary name for this container. Note the '.', which tells docker where to find the Dockerfile. You've probably noticed I've run 'docker build' without 'sudo': for practical purposes I've added docker into the sudo group.

Now, if you want to test quickly you can use this Chrome extension, which provides a GUI to instantiate a WebSocket connection and send and receive data through it, and play with it. The URL will be: 'ws://IP_ADDRESS:8080'.

You can also access the server's logs with:

docker logs 6ce3498a67e2

(where 6ce3498a67e2 is the first part of the container's unique identifier, as shown in the 'docker ps' output).

Once you have this in place, which takes much longer to describe than to do, you can start building your WebSocket server logic.

Sunday, 11 January 2015

and each architecture needed to be replicated on a number of "logical" environments (e.g. 'development', 'testing', 'production').

They needed to "see" each other, in a secure way.

Note that virtual machines on Digitalocean (they call them 'droplets') can belong to different data centers.
When the droplets use the optional private interface there are two things to consider:
1. Traffic inside the same data center is potentially visible to any equipment on the same data center.
In other words, the fact that two droplets belong to the same customer account doesn't mean that their private traffic is isolated from any other traffic belonging to droplets on other accounts.
You are responsible to secure that traffic.
2. Droplets on different data centers cannot directly communicate via their private interfaces.
This is intuitive but nevertheless important to highlight.

This of course screamed for a VPN, and I directed my attention to openvpn.
As for the rest of the system and application setup I wanted to use Puppet to manage the client and server configuration with openvpn.

A quick search on Puppet Forge brought me to luxflux' openvpn module, which at that moment was the most popular.
I experimented a little with it and found it working properly, on the debian virtual machines I was using.

You need just some designing of the private network addressing, and you'll be fine.

The key point to use this module is understanding that each VPN client you declare on the VPN server will cause the generation of client certificates on the server machine.
You then need to copy them into the VPN client hosts.

On the client side I've created a new Puppet module.
It just installs the certificates and keys generated by the openvpn module running on the server host, then instructs the openvpn client to connect to the remote VPN server.
(This is a fairly generic module, so it could probably be open sourced).

To make the architecture easier to understand, I assigned a different subnet for each "real" data center. So, even if the clients all belong to the same VPN, at a glance you can see what's the hosting data center.

The VPN server is on one of the data centers, and clients can connect securely on the public interface. An additional level of security was to firewall access to the openvpn listening port (with iptables), so that only authorized source IP addresses could potentially be used to build the VPN.

During this work I submitted a few patches, which the author has kindly accepted to merge:

1. Add an optional parameter to enable the client-to-client option
2. Add optional parameters to enable the openvpn management interface
3. Add an option to remove a client-specific configuration file (for example when a client is removed from the VPN).