Ruby + SOAP4R + WSDL Hell

I’ve been spending a bit of time lately playing around with Ruby on Rails the last couple days, giving it a bit of a test to see what everyone’s raving about and understand how it could be used. Overall it’s a pretty impressive framework, and worth a little time investment to give it a whirl. For those without the patience, I highly recommend viewing the impressive presentation that demonstrates development of a blogging tool in 15 minutes.

That said, all is not sunshine and chocolate in the world of Ruby. I spent the better part of today trying to get some basic SOAP functionality operating to allow me to interact with Amazon Web Services’ E-Commerce Service. I was using Hiroshi Nakamura’s soap4r library to auto-generate Ruby class definitions from a WSDL file, but I was running into a bit of pain. There seems to be next to no information out there about how to use soap4r and the associated wsdl2ruby class generation utility, and even less about the current shortcomings of the current stable release of the library. In the interest of saving someone a day of time, I thought I’d put together some details about the wsdl2ruby tool, the files it generates, and what does and doesn’t work.

Just a disclaimer: I’m no whiz in the whole SOAP/WSDL arena, but I think the information I’m about to provide you with will be enough to help you figure out what’s going on when using soap4r. Your mileage may vary.

Generating Classes with wsdl2ruby

To create applications capable of accessing web services via SOAP, you could compose raw SOAP requests yourself (see the â€œBehind the Screensâ€ article for more detail), but that would be a bit painful and require a fair amount of manual labor. I’m a lazy, lazy programmer, and I’m betting you’re the same.

A better approach is to use a framework that can automatically generate class definitions for a framework that can be used to create objects, map those objects to SOAP, and vice-versa. This is exactly what soap4r and the wsdl2ruby provides. Using soap4r, a developer can easily generate both client and server classes to handle consuming and providing SOAP-accessible services. For my purposes, I’m only interested in generating client code to allow me to develop an application that can consume services.

The wsdl2ruby application does exactly what it name implies: it takes a Web Services Description Language definition of a web service, and transforms it into Ruby code. For this exercise, I’m going to use the 2006-03-08 WSDL definition of the Amazon Web Services’ E-Commerce Service web service available here.

To generate client code for the Amazon.com web service from the WSDL description, run wsdl2ruby like this (your platform may require the path to be set appropriately):wsdl2ruby.rb --wsdl http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl --type client --force

This command will generate three files:

AWSECommerceServiceClient.rb: An example client that provides skeleton code for exercising the web service. This code can’t really be run â€œout-of-the-boxâ€, something I’ll talk about in a moment.

default.rb: The set of class definitions for all elements defined by the WSDL file. Using these class definitions, a developer will be able to produce and consume the various building blocks required to interact with the web service without needing to search an XML tree, or perform any other similar ugliness.

defaultDriver.rb: This file contains a single class, AWSECommerceServicePortType, which is used to conduct all requests of the web service.

Although I won’t be doing it for this exercise, you could easily rename default.rb and defaultDriver.rb as you see fit; however, you’ll have to make sure to update any require statements to reflect your new naming.

Notice that body is set to nil, whereas the web service requires an ItemLookup object to work. To make this code work, you’d need to create an ItemLookup object – as it turns out, ItemLookup relies on ItemLookupRequest, so you’ll have to create one of those as well. The parameters required to create these objects are determined by the WSDL definition, and the order of parameters to pass to new are documented in part in default.rb; the meaning of those parameters are given in the Amazon Web Services’ E-Commerce Service API documentation.

As an example, let’s say I want to perform a simple lookup for an item with an ASIN of B00005JLXH – which just happens to be the unique Amazon identifier for Star Wars, Episode III (it was the first thing I saw on the Amazon home page, I swear). First I create the specific ItemLookupRequest object for that item:

Note that the class constructor generated by wsdl2ruby requires all parameters to be specified (their default value is nil), so you need to provide all the parameters. Use empty strings for the ones you don’t need or want to provide.

Next, I create an ItemLookup object, adding both my developer token and the ItemLookupRequest object I created above.

Finally, I call the itemLookup method on my AWSECommerceServicePortType instance to execute the call to the web service, and print some output using the resulting ItemLookupResponse object as the source of the response data:

Once the request completes, the itemLookup method will return an ItemLookupResponse object, which will allow you to programmatically access all of the returned data in a simple programmatic fashion via the accessor methds generated by wsdl2ruby. In theory.

In theory, Communism works. In theory.

Unfortunately, wsdl2ruby is still seems to be a work in progress, and therefore doesn’t work as cleanly â€œout of the boxâ€ as one might like. For one thing, it seems to have some difficult with complex types. The example above will undoubtedly choke with something like:

This problem arise from the lack of a class name being generated for the OperationRequest associated with an ItemLookupResponse. As defined by the WSDL, an ItemLookupResponse is defined as:
<xs:element name="ItemLookupResponse">
<xs:complexType>
<xs:sequence>
<xs:element ref="tns:OperationRequest" minOccurs="0"/>
<xs:element ref="tns:Items" minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
</xs:complexType>
</xs:element>

The problem here is the two highlighted nil class names generated by wsdl2ruby. The ClassDefCreator class in soap4r is responsible for generating this class definition – I took a lookup at the definition and found the following issue in dump_classdef (follow along in your own install of Ruby, in {ruby install path}lib/ruby/1.8/wsdl/soap/classDefCreator.rb):

The problem is that last type = nil statement. Basically, if ClassDefCreator encounters a complex type in the WSDL, it assigns a nil type. What’s odd is that there appears to be a perfectly good solution currently commented out of the code. If we change:

Now, the soap4r framework will be able to find the appropriate class to use to represent the OperationRequest when transforming the response SOAP XML into a object. I’m a little puzzled why this fix is currently commented out in ClassDefCreator – I assume there’s probably a good reason. In all likelihood, this solution is probably commented out because the class name alone isn’t enough to avoid namespace clashes. I’m sure for a more complicated application consuming several web services this would undoubtedly be an issue, but for my purposes, this is not an issue.

tricks wsdl2ruby into generating the correct Header class definition. As this similar construct exists throughout the WSDL, I performed similar changes throughout – primarily I changed the Arguments WSDL definition to make sure an Argument class is properly generated.

With those changes in place, regenerate the class definitions as before, and run the AWSECommerceServiceClient. This time it should provide the desired output:
ASIN: B00005JLXH
Detail Page URL: http://www.amazon.com/exec/obidos/redirect?tag=ws%26link_code=sp1%26camp=2025%26creative=165953%26path=http://www.amazon.com/gp/redirect.html%253fASIN=B00005JLXH%2526tag=ws%2526lcode=sp1%2526cID=2025%2526ccmID=165953%2526location=/o/ASIN/B00005JLXH%25253FSubscriptionId=0VS96BNQBVY904T3XZ02
Title: Star Wars, Episode III - Revenge of the Sith (Widescreen Edition)

Ah, Closure

Lesson of the day: my pain is your gain. While the wsdl2ruby utility is not fully baked to handle the full flexibility provided by WSDL, it can be cajoled into doing the right thing to get the results you desire. Although I was able to get my simple example working, I’m sure there’s any number of esoteric cases that the soap4r libraries don’t currently handle. Until they do, you’ll have to tinker a bit to get them working. Good luck!

# This program is copyrighted free software by NAKAMURA, Hiroshi. You can
# redistribute it and/or modify it under the same terms of Ruby’s license;
# either the dual license version in 2003, or any later version.

Hey John – I had the same question – I believe that SOAP4R has been integrated into the standard Ruby distribution, but I found that the version that accompanied Ruby didn’t seem to work (I can’t recall the issue). I assume they’re the same library (or rather, probably different versions of the same library) – I’m not a big Ruby expert, so the issue I encountered with the version that accompanies Ruby might just have been an issue with my environment.

JohnJune 9, 2006

Thanks for the great walk-through. One problem I had though, was that charset.rb was incorrect in the soap4r package version 1.5.5. The error was converting ISO### to UTF8 and it’s noted here http://dev.ctor.org/soap4r/ticket/172 and http://dev.ctor.org/soap4r/changeset/1656 as a mispell. I don’t understand why or how, but I checked tar file and the changes noted weren’t in the file I just downloaded. Anyway, I had to change that code specified. And later I just downloaded and installed the entire source from subversion.

also I had to look up the methods, for me most had to be lower case, items, item, asin, etc
itemLookupResponse.Items.each do |item|
item.Item.each do |innerItem|
puts “ASIN: #{innerItem.ASIN}”
puts “Detail Page URL: #{innerItem.DetailPageURL}”
puts “Title: #{innerItem.ItemAttributes.Title}”
end
end

I did not have all the other problems mentioned in this post, but this post helped so much getting started.

anneJuly 13, 2006

also, I was told to ignore those two messages at the beginning
Exception `LoadError’ at
/opt/local/lib/ruby/1.8/xsd/xmlparser/xmlparser.rb:10 – no such file to
load — xml/parser
Exception `LoadError’ at
/opt/local/lib/ruby/1.8/xsd/xmlparser/xmlscanner.rb:10 – no such file
to load — xmlscan/scanner

KeithJuly 17, 2006

I’m executing the wsdl2Ruby.rb and nothing happens. No classes get generated and I am confused as to why this would happen. I installed ruby, the soap4r, and the http-access2. When I run the command, wsdl2ruby.rb –wsdl http://webservices.amazon.com/AWSECommerceService/AWSECommerceService.wsdl –type client –force , or any wsdl the program runs and outputs nothing. No classes or words to the prompt.

I also went ahead and placed the wsdl in the same directory as wsdl2ruby, I ran this and nothing happend. Do you have any ideas?

SanjeevAugust 25, 2006

Keith, I am encountering the same issue where no files are generated. I am on a windows platform. Have you been able to resolve this issue?

Anyone have suggestions?

I noticed on the soap4r site that there is dependency on UCONV. However, I have not be able to find a windows version. The readme file suggests that one should define USE_WIN32API for a windows env. However, it is not clear where this needs to be definded.

In response to Keith (July 17th), I had the same problem. I ran install.rb, and everything appeared successful. But, wsdl2ruby.rb did not do anything. I could not debug it either/

The answer is that there are three wsdl2ruby.rb files. The installer installs the /lib and /source files, but it did not install the third file. This is the crucial one. It should be located in /ruby/bin.

How do you fix it? Open the soap4r gzip file. Look at the /bin directory. Copy the wsdl2ruby.rb file to the ruby/bin directory.

One strange thing I found is that wsdl2ruby wasn’t recognizing anyType as a valid type. I had to get around it by modding ClassDefCreatorSupport.rb … Seems to work, as ugly as it is.

def basetype_mapped_class(name)
# I MAY GO TO HELL FOR THIS HACK
return name if name == XSD::AnyTypeName
# END HACK
::SOAP::TypeMap[name]
end

DaveOctober 16, 2006

I also encountered Mark’s problem (with a different wsdl) and it seems that wsdl2ruby can’t handle the union in the definition of lang. I was able to work around this by downloading the wsdl and hard-coding the namespace schemaLocation reference to the 2004/10 version of that schema.

For me it was changing this:

to this:

(note – the 2001/ url always points to the latest revision of the xsd – currently 2005/08)

Thomas L. KjeldsenOctober 25, 2006

You might want to check out this article at linuxjournal: http://www.linuxjournal.com/article/8969 – see “Listing 6. loc_service.rb” where SOAP::WSDLDriverFactory is used to create a dynamic proxy based on a wsdl description.

This article is super helpful – thanks. I noticed a couple of issues in implementing the Amazon system.

First, I’m Windows XP and I had to install Soap4r from the zip file (http://dev.ctor.org/download/soap4r-1.5.7.zip) NOT from the Gem. The Gem seems to install itself in a load order that doesn’t work. Also, you have to install http-client FIRST (gem is ok) before installing Soap4r manually. Manual install was super smooth – but you might want to make sure you have a Windows version of Gnu Win util’s “chmod.exe” just to avoid error messages (don’t think it does much for us).

After solving that little issue, there seems to have been some changes to Amazon WS since you wrote the article.. Here’s the code that I ended up writing to put in AWSECommerceServiceClient.rb (around line 30). Basically, you refer to “.Item” and now should be “.item” and you refer to “.ASIN” which is now “.aSIN”. Also “.ItemAttributes” => “.itemAttributes” and “.Title” => “.title”

name spacing and xsi:type is not correct
in the body and Envelope tag is env: it should be soap: in my case also
the Procedure being called must be prefixed with tms: and the arguments must have a xsi:type=”xsd…..”
namely the missing xsi:type is going wrong here.. how can I add this. I used wsdl2ruby to create the objects…

Hmm, sorry Roman, it’s been so long since I did anything with this code that there’s no tips I can really offer on what might be causing this issue. You might post to the soap4r Google Group to see if anyone there has had this problem.

RomanJuly 26, 2010

Hi Brendon, I discovered the problem lay with invoking the client with the -d flag of all things! In any case, thanks again for the excellent tutorial!