The Evolution of SOAP::LITE

I worked recently on a design of a lightweight Internet-based payment
system using SOAP, and after having done so, I learned a number of
valuable lessons that I thought I'd pass on here. First I'll talk
about why I chose this route, then I'll talk a little bit about
recent changes in the SOAP community, including the directions that
this technology appears to be moving.

Based on what I learned while designing a similar payment system last year, plus already knowing little bit about SOAP, I thought that SOAP would play pretty well for the new system I was working on. I built the prototype in Perl, so I took a look at Keith Brown's SOAP/Perl module. After a couple of experiments, I was able to make my first SOAP call.

Even though my first impressions of these new tools were good, using the SOAP/Perl library initially wasn't that easy for me. So I wrote a simple wrapper around Brown's class that cut down the number of lines required to make a call on the client side from 25 to 5. The lesson that I've learned over the years is that simplicity and flexibility are the keys to progressing up the knowledge curve quickly. In that spirit, I decided to keep the interface as simple as possible on both the client and server sides.

Why SOAP?

Why SOAP and not something else?

I don't have extensive experience with XML-RPC, and I knew that
relying on RPC calls only wouldn't be adequate for payment system I
was working on. I realized that at some point I would need to send an
XML document. That's why I took a closer look at SOAP.

Another appealing aspect was its ability to run on top of multiple protocols. I had planned to provide several ways for message delivery. Initially, SOAP's ability to extend functionality with headers looked more theoretical than practical for me, but as it turns out, it's very handy.

In the very beginning, it was just a brain exercise to build module that would serialize Perl data structures into XML, then deserialize it back into the Perl data. There were several modules on CPAN that could do similar things at that time, but I was looking for a module that could generate XML according to the SOAP specification.

Are you involved in SOAP development? If so, let us know what you're working on.Post your comments

After two days of thinking about this, and six hours of programming, I ended up with the module that could serialize the data and make a simple SOAP call. The first version had 200 lines and no deserialization part. One week later, I had a fully functional module that was about 600 lines and passed the first tests with public servers. I thought that two more weeks would give me all functionality I needed. I was naive, and eight months later I'm still not finished. But in the meantime, many new developments have emerged in the SOAP community.

SOAP comes of age

Three months into my project, I began to notice lots of activity in
the SOAP community. Module grew in functionality and in number of
available transports, such as SMTP, POP3, TCP, FTP and IO.
Implementation of a pluggable serializer and deserializer allowed me
to add support for XML-RPC protocol and to reuse all available
infrastructure. Following the motto "Run everywhere where Perl runs,"
regexp-based XML parser was added to provide fallback on platforms
where XML::Parser is not available (like Windows CE) [1].

This year most of the activity in SOAP was related in one way or another to achieving interoperability. The number of available SOAP toolkits are growing really fast -- more than 65 different ones are available, and the vast majority of those toolkits interoperate pretty well.

You should have no problem finding toolkits for the languages that you like to work with. Toolkits for more than 15 languages are available (Perl, Java, Python, C++, C#, languages with COM interface, Ada, JavaScript, PHP, Tcl, Ruby, Smalltalk, Delphi, XSLT, Orchard, K) with more than 10 toolkits for Java and 5 for C++. The choice is yours [2]. Authors of about 25 implementations actively participate in the
process of improving
interoperability between their toolkits.

At the same time, there's still lots of work to do. We are moving through the stage of achieving basic interoperability on several levels right now:

Transport -- including proper status codes for HTTP, handling character sets in "content-type" headers, the SOAPAction header, and others.

SOAP -- RPC-style communication, support for attributes, like actor and mustUnderstand).

WSDL -- because many toolkits rely on service description support for making and receiving SOAP calls.

The direction SOAP is moving

More complex systems are being built on top of these basic levels of interoperability. SOAP development is moving in two orthogonal directions: creating extensions using SOAP extensibility and implementing complex conceptions.

The most important extensions that will influence future of SOAP are: SOAP Messages with Attachments (describes how to wrap SOAP message into MIME envelopes together with various attachments, like images or binary objects), SOAP Security Extensions: Digital Signature (describes how to implement XML Signature specification in SOAP to add security features on protocol level), and encryption (based on XML encryption, that brings other security features, like confidentiality to the SOAP level).

Complex implementations include routing through several SOAP processors (intermediaries), reliable messaging, and asynchronous processing are also planned. Recent specifications from Microsoft and IBM are trying to address these aspects. SOAP-RP from Microsoft describes message paths that two-way message exchange, and WSFL from IBM defines usage and interaction patterns for web services.

Looking at the SOAP::Lite module, you may find some similarities with Perl itself. You can write a useful program in just three lines of code. There is also the "more than one way to do it" flexibility with SOAP that we see in the Perl community. And even though SOAP appears simple on the outside, there's lots of power under the hood.

One of the things I enjoy about writing SOAP modules is to find existing resources on CPAN that I can incorporate into my work. It allows me to prevent code duplication, and reuse tools such as the XML parser, transport modules, compression, MIME generation, and many others.

What is on the table for the future? MQSeries and Jabber transports,
non-blocking TCP and HTTP servers with SSL support, asynchronous
processing, lightweight DBD implementation for remote database access
on top of SOAP/XML-RPC, new sections for SOAP Cookbook, Quick
Start Guide, and experiments with optimization technique that
could bring significant performance benefits for already parsed
messages.

That's only the major stuff. The entire TODO list is five
hundred lines long. Any volunteers to cut it down? Watch for the new
versions and come to talk during the upcoming Open Source Convention.

Pavel Kulchenko
was a speaker at the O'Reilly Open Source Software Convention and has over 10 years of experience in design and
development of complex financial applications.