Category Archive

I’ve been half heartedly seeing if any softphone makers are interested in supporting the pseudo ICE mechanism that GTalk uses when setting up the media on an XMPP call. It doesn’t look promising at this stage. Google did make a change around the start of this month to their XMPP call set up mechanism, which broke the ability for Asterisk 1.8 to place outgoing calls through GTalk, so maybe they are working on the service and will have full Jingle support soon which in theory would allow ICE compatible phones such as Counterpath’s softphone range and others to be able to place SIP calls through sipsorcery and have them terminated via GTalk/Google Voice’s XMPP/Jingle service. That would be neat as it would be a validation of SIP and XMPP signalling working together and interconnecting two technologies which both support a large number of users.

However Jingle for GTalk isn’t here yet and in the interests of encouraging any developers that are involved with writing softphones to look at supporting the Google STUN requests on the RTP sockets I’ve created a prototype application that shows how to do it. The application is written in C# and hosted on codeplex here. What the application does is listen on a socket for a SIP INVITE request and when it gets one translates it to an XMPP request which it sends off to GTalk. As well as handling the SIP and XMPP signalling the prototype application also fires up two media sockets, one that talks to the Google XMPP end and one that talks to the SIP phone end. The media sockets are needed so that the STUN requests and responses required by the Google XMPP end can be handled correctly and that’s the bit that’s missing from the softphones.

I’m not having much luck getting the counterpath softphones, either Bria 3 or XLite 4, to work with the Google Voice media server. The problem is the counterpath softphones don’t seem to be implementing ICE properly or perhaps are implementing an earlier version. The specific issue is they don’t send a STUN binding request on the RTP socket and instead launch straight into sending their own RTP stream. The Google Voice server isn’t interested in RTP until it’s done the STUN binding request exchanges. Update: it was my own malformed SDP packet that stopped the XLite from working properly. Once that was fixed the XLite did send a STUN request as part of the media initialisation. The problem then became the Google Voice server not recognising the STUN request because it implements an earlier version of STUN/ICE and it doesn’t recognise the STUN attributes set by the XLite.

I tried out the free version of the Zoiper softphone as well but it doesn’t look like it support any version of ICE so I didn’t get anywhere with it either.

If anyone is aware of a SIP softphone that supports ICE please let me know.

Update: Thanks to Avi Marcus I tried out the Blink softphone which like the XLite also supports ICE. It also send the STUN binding request to initialise the media and as with the XLite it’s requests were rejected as malformed by the Google Talk/Voice XMPP server due to the newer STUN attributes.

Unless something else crops up I think that’s pretty much the end of any attempt to integrate Google Voice XMPP calls with sipsorcery for a while. To work around the mismatch in STUN/ICE versions between softphones that do support ICE and Google Talk/Voice would mean proxying the media and that’s not something the sipsorcery service is geared for. The Google Talk developer document does state they intend to implement the latest versions of Jingle and that would mean also supporting the latest version of ICE. When that happens there will be more options to translate between SIP and XMPP with Google Talk/Voice.

It’s been a while, at least 5 years, but I’ve eventually ended up back in the bottomless pit of frustration that is javascript. Actually it’s not that bad these days thanks to jquery although it’s still far from what I’d call pleasurable. It wasn’t an entirely voluntary decision for me, my day job requires me to get down and dirty with a web application and that means javascript and jquery.

Anyway I have whipped up a quick little sample to query the SIP Sorcery provisioning service to retireve a list of SIP accounts using jquery. It might be useful to someone and as with the previous samples if there is any interest I’m happy to flesh out the remainder of the REST interface to make it usable.

I had a question on the forums about creating a PC based widget to display the callerID on an incoming call by maintaining a persistent connection to the sipsorcery telnet monitoring server. One problem with the design is that sipsorcery no longer uses telnet the end of telnet for sipsorcery, so the design would need to change to use SSH instead. The question mentioned using Adobe’s AIR to build the application and while I don’t know a lot about AIR it’s likely that a telnet client would be easy to implement, since it’s not much more than a TCP socket, but SSH would be a lot harder as there’s a lot of extra work to negotiate and set up the encrypted channel.

As it happens at the same time the SSH change was made I changed the way the Siverlight client retrieves the notifications messages from the server from using a telnet connection to a HTTPS one. Part of the motivation was exactly the same as the problem in the previous paragraph, it was going to be tricky to implement an SSH client in Silverlight. Another reason was that the Silverlight client can only establish TCP connections to destination ports 4302 to 4332 and I had begun to find that a bit frustrating, I couldn’t use the console from a previous workplace, some internet cafes etc.

The HTTPS service that the Silverlight client uses to get notifications is . The “pull” at the end of the service URL is pertinent and the notifications mechanism requires that clients poll the sipsorcery web server to pull notifications down. Such a pull mechanism is not ideal as it means unneccessary traffic and load but after spending the best part of two weeks fighting with Microsoft’s PollingDuplexHttpBinding only to conclude that it’s completely broken with IIS6 and switch back to a traditional pull mechansim. Maybe at some point I’ll revisit it, the sipsorcery web site has since moved to Windows Azure which uses IIS7 and on another front HTML5 has introduced Web Sockets which achieve the same as the PollingDuplexHttpBinding.

The ability to pull notifications from the sipsorcery web server is something that can be used right now to build an application like the callerID widget. The Silverlight client consumes the service using a WCF SOAP endpoint. Connecting to WCF SOAP endpoints is ok if you’re going to be writing a client in .Net (C#, VB.Net etc) but it can get tricky from non .Net languages especially when the service involves authorisations which the sipsorcery one does. One great thing about WCF is that with very little effort a service endpoint can be added to support an alternative interface. In this case REST and JSON (I’m not deliberately trying to set a record for acronyms) are a lot more universally understood compared to SOAP and .Net’s XML serialisation. I’ve spent a few hours adding the REST and JSON interface to the notifications service and have deployed it to the sipsorcery Windows Azure web site, the URL is https://www.sipsorcery.com/notificationspull.svc/rest/. The service interface is:

It’s always been on my todo list to spend more time with Ruby with a big motivation being able to write some more powerful sipsorcery dialplans. Last week I got an incentive to finally do it when I came across a Ruby on Rails cloud platform called Heroku. Heroku operates on top of Amazon’s EC2 but rather than dealing with EC2 instances and images they abstract that all away and allow programmers to work at an application layer. The Heroku experience is close in some respects to Microsoft’s Windows Azure cloud platform, which I’ve also been spending a fair bit of time with. There’s pros and cons to each of course, Heroku’s is a lot more polished, a lot quicker to pickup and a lot quicker to deploy with while Windows Azure is more powerful given that the .Net framework is more comprehensive and has deeper hooks into the underlying Windows OS than the Ruby on Rails software does. In fact it would probably be a better comparison to compare an ASP.Net MVC web application hosted on Windows Azure to Heroku’s offering. The same pros and cons listed previously still apply and for a programmer unfamiliar with either Heroku and Ruby on Rails have a much smaller learning curve but in my opinion an ASP.Net MVC application is more powerful and manageable.

What’s all this got to do with sipsorcery?

As part of my exploration of Heroku & RoR I thought it would be worthwhile to get a basic web app deployed that would hook into the sipsorcery web services. It would give my exercise of attempting to learn RoR a goal and maybe it could be the starting point for another programmer out there to put together a non-Silverlight interface for sipsorcery, something which comes up regularly on the sipsorcery forums.

The sipsorcery web services have always been available via a SOAP API although as far as I know nothing except my Silverlight client has ever consumed it. SOAP APIs are out of fashion these days though and REST and JSON are now the flavour of the month, they are also easier to use so that’s good news. I have exposed a few sipsorcery methods over REST, specifically the login and get SIP account methods, and it was there that I wanted to hook up from a Heroku RoR app.

The result of my efforts is at https://sipsorcery.heroku.com/auth/login. It’s a completely bare bones application, absolutely no effort has gone into the user interface, that lists the ID’s, usernames and domains of the first 10 SIP accounts belonging to a sipsorcery user. I’ve only been using RoR for a week so I’m not fully confident about the application being fully secure but the two communications channels, browser to the Heroku app and the Heroku app to the sipsorcery REST API are both SSL, so I’m satisfied enough to use my own sipsorcery username and password with it.

If there are any RoR programmers out there that use sipsorcery and are interested in a non-Silverlight interface my very crude code for the above application is hosted on github.

The main concept of the RoR app is the connection to the sipsorcery REST API. The Ruby code to accomplish that is very succinct and I’ve posted it below for anyone interested. The user and pass values need to be replaced with valid sipsorcery account details and if the code works it will dump SIP account details as a block of JSON encoded data which looks very messy but is easy for a programmer to work with.

This post is aimed at anyone thinking about contributing to the sipsorcery code base. The code base is written in C# and heavily utilises Microsoft’s .Net framework. If you’re not familiar with C# or .Net that’s not necessarily a big problem. To contribute there are two main attributes you need:

A good understanding of SIP and familiarity with the sipsorcery or mysipswitch services,

A good grasp of programming fundamentals. The language is not that important but a background in any of C, C++ or Java is useful, but not essential, as they are all pretty close to C# syntax wise.

If you have those two attributes and want to help or are frustrated that a fix you want is not getting done fast enough then read on.

Quicksteps

A minimal set of steps to get you to a point where contributiions can be made are:

To contribute is as simple as uploading a patch. To do that you’ll need a codeplex account after which you can upload any sipsorcery code files you modify using the Upload Patch function. When patches get uploaded I, and hopefully in the future other sipsorcery project coordinators, will have a look over the patch and if it’s ok commit it to the source tree.

For regular contributors instead of uploading patches and requiring a coordinator to check and commit a better option is to become a project developer and commit changes directly using Subversion. Before going down that path it’s preferred that you upload a couple of patches so that we can check that you know what you are doing and aren’t going to blow anything major up.

Building the source

To build the source code you’ll need two things:

A copy of the source which can be obtained using Subversion or downloaded as a zip file from the sipsorcery project Source page,

A C# compiler. A command line C# compiler is available free from Microsoft but most people build their .Net projects using an IDE which includes the compiler. The advantage of the IDE is it makes all of the build tasks simple point and click operations. The most popular .Net IDE is Microsoft’s Visual Studio .Net. The current version is Visual Studio .Net 2008 and it’s the version the sipsorcery project files are formatted for (Visual Studio .Net 2010 is in beta and I haven’t tested it out as yet). The full version of Visual Studio .Net costs a couple of thousand dollars but luckily Microsoft provide free cut down versions which are pretty much feature complete except for some more advanced and less commonly used functions. The free Visual C# 2008 Express Edition is the recommended tool for anyone wanting to start out with the sipsorcery code base.

Once you have an IDE installed and downloaded the source tree the solution file most people will be interested in is in the sipsorcery-core directory and is SIPSorcery-Core.sln. Double clicking on that file should open the solution in your IDE and to build simply press F6. If everything goes according to plan the build will be successful and you will now be ready to start thinking about making changes.

Making Changes

The main thing most people will be interested in fixing or enhancing are dialplan functions. In recognition of that the dialplan functionality was placed into a separate assembly (an assembly is what a library or component is called in the .Net World) called SIPSorcery.AppServer.DialPlan. Within that project a most of the dialplan functions exposed to the sipsorcery Ruby dialplans are contained in DialPlanScriptHelper.cs. If you want to fix something in that class you need to make your change, check the project builds and then upload the file(s) you have changed on the sipsorcery site Upload Patch.

If you hit any snags please feel free to ask for help in the comments.

One thing I’ve been meaning to do for a while is a post on how to programatically connect to the SIP Sorcery provisioning service. The service is exposed over a SOAP 1.1 interface. This post provides a brief C# code example which demonstrates how to connect using WCF. At some stage I’d also like to provide a javascript based code sample using jquery or a similar library. That may motivate someone to write an alternative user interface to the Silverlight one and appease the people who dislike it for whatever reason.

The SOAP standard does not include any mechanisms for authentication. There is a web service extension available that includes a mechanism in the form of WS-Security specifications. The problem is that the specification is quite bulky and the classes provided by WCF to make it easy to use are not supported by Silverlight.

An alternative to using a SOAP authentication mechanism is to use an HTTP one such as OAuth. The HTTP approach is appealing for at least two reasons: it’s light weight and easy to implement compared to WS-Security and it can be used for different application protocols which means if/when a REST provisioning endpoint is added the same authentication mechanism can be used.

Incorporating REST and OAuth into the SIPSorcery provisioning interface is a little way down the road and at the moment the authentication mechanism used has been designed to work easily with the Silverlight client. The existing mechanism is SOAP specific and involves two steps:

Step 1 – Call the Login method and if successful a token will be returned. The token is a 384 bit random number returned as a 96 byte string.

Step 2 – In all subsequent provisioning SOAP requests the token needs to be included in the SOAP header in an authid element.

Each token is only good for one hour or until it is passed to the Logout method. While the token is current it’s a critical piece of information that allows full access to a specific user’s account as such it should never be transmitted over an unencrypted connection. The SIPSorcery provisioning interface is only exposed via HTTPS which means there is no opportunity to send a token on anything but an encrypted connection.

Below is an example of a SOAP envelope that includes the authentication token in the authid SOAP header. The SOAP request in this case is GetCustomer and the username parameter in the body is actually the username of the customer record being requested and nothing to do with authentication.

All but one of the classes for the returned objects are contained in the SIPSorcery.SIP.App assembly and the SIPAssets folder. The SIPAccount class for example.

The exception is the Customer class which is contained in the SIPSorcery.CRM assembly.

Finally we would request that the interface is used sensibly. It can be used to create new sipsorcery accounts (when they are enabled again) and as they are in tight supply there may be a temptation to automate their creation. At this point we do request users stick to one account each and while we are very reluctant to suspend or remove accounts and only do so as a last resort if one user’s actions have a large impact on everyone else we will do so.

Enjoy and plese post a comment if you are successful in connecting to the provisioning service.

The Silverlight UI that has been employed on sipsorcery.com and which replaces the AJAX UI on mysipswitch.com has caused some serious gnashing of teeth. The two reasons I have been able to distill for the frustration seem to be no Silverlight plugin for browser xyz or OS xyz, which is a fair point, or secondly a dislike of anything Microsoft and the hassle of downloading another plugin. Both those arguments and lots more about the pros and cons of different browser technologies are prolfigate all over the web so I won’t bore you with my own.

The purpose of this short post is instead to explain why the AJAX interface was replaced by Silverlight. There are two reasons:

I really really dislike javascript/DHTML programming. It’s incredibly frustrating to switch from a sophisticated IDE and compiled code language such as C# (or Java if you’re that way inclined) back to fiddly little HTML tags and a hodge podge of javascript libraries and browser hacks which is otherwise know as AJAX (Asynchronous Javascript and XML). Some programmers thrive on AJAX, I’m not one of them.

Silverlight has this massive thing under the hood called the Common Language Runtime (CLR). The CLR is what runs the latest version of software developed on Microsoft’s .Net platform. The Silverlight CLR that runs in a browser is a cut down version of that runs on a desktop but it’s still suprisingly comparable. In contrast to AJAX development I find C# development to be the bee’s knees and makes programming fun rather than like putting hot pokers in my eyeballs. Because of the CLR a Silverlight application can also share code with non-Silverlight applications. In the case of sipsorcery the really big thing is that the SIP stack which drives all the servers can actually run in the browser. What that means is some very cool SIP applications can be developed.

In answer to a question about whether the sipsorcery UI could be targetted to the Silverlight 1.0 runtime so that it would run with Moonlight (the Linux port of Silverlight) the answer is unfortunately no. Version 2 of Silverlight is the first one that included the CLR and that’s the whole point of sipsorcery using Silverlight.

It’s now been 3 weeks since the Isolated Process dial plan processing mechanism was put in place on the sipsorcery service. The news on it is good and while there were a few tweaks required in the first couple of weeks, which were more down to preventing some users initiating 20+ simultaneous executions of their dialplans, in the last week there have been no software updates or restarts required. During that time the sipsorcery application server, which processes the dial plan executions and has been the trouble spot, operated smoothly with no issues.

As discussed ad-nauseum in the past the root cause of the reliability issue on the services is a memory leak either in the Dynamic Language Runtime (DLR) or in the integration between sipsorcery and the DLR. The solution has been to isolate the processing of the dialplans in separate process and perioidcally recycle those processes.

I now feel pretty comfortable about the reliability of the sipsorcery application server and am reasonably confident that a solution to the instability issue that has plagued mysipswitch and sipsorcery has been found, at least for sipsorcery. As also mentioned previously the mysipswitch service cannot be easily updated anymore since the code has diverged significantly since it’s last upgrade in November of last year. I would now recommend that people migrate from mysipswitch to sipsorcery for greater reliability. There were two cases where the mysipswitch service needed to be restarted in the last week due to the “Long Running Dialplan” issue and a failed automated restart. On average the mysipswitch does need one restart a week. If the restart happens to coincide with times when I or Guillaume are able to access the server, which is when we are not asleep and in my case at work, it’s fine. If it’s outside those times it can be up to 8 hours.

Update: Of course no sooner had I posted about stability there was a problem. Approximately 5 hours after posting the above the dial plan processing on the Primary App Server Worker failed with calls receiving the “Long Running Dialplan” log message. The memory utilisation of the App Server was low, around 120MB, and the process was responding normally, if it was not the Call Dispatcher process would have killed and recycled it. The thing that was failing was script executions by the DLR. This provides some new information and it now looks like there are two separate issues with dialplan processing. One is a memory leak when a process continuously executes DLR scripts. The second is a bug in the DLR that causes it to stop processing scripts altogether and possibly the result of an exception/stack overflow in a script. The memory leak issue has been resolved by recycling the App Server Workers when they reach 150MB. An additional mechanism is now needed to recycle the process if script executions fail.

The version 1.1 release of sipsorcery has been made and can be downloaded from codeplex. While there have been a month’s worth of minor fixes since the v1.0 release the main reason for the v1.1 release is so anyone interested can use the GoogleVoiceCall application in their dialplans.