Chapter 15: An XML-RPC Service

An XML-RPC Service

This chapter demonstrates how to build an XML-RPC service in Rails using ActionWebService. Like the rest of this book, this chapter won’t simply drive through the mechanics of setting up a service. Building on Chapters 11 and 12, we’ll learn not just how to make a Rails app communicate with another Rails app—that’s easy—but also how build an infrastructure for services that makes your application enterprise-solid. That involves properly abstracting your database schema—the physical data model—from your logical model. Enterprise-solid also means knowing where to put all of your service code so your application remains coherent. But first, you need to get set up with ActionWebService, which can be tricky in Rails 2.0.

ActionWebService and Rails 2.0

As noted before, in Rails 2.0, ActionWebService was relegated from the core to plugin status. As of this writing, simply installing the ActionWebService gem does not work as expected with the following:

gem install ActionWebService

You may want to try installing the gem as above. The warning sign that the plugin is still not fully integrated are errors such as “Uninitialized constant ActionWebService”. If you want to wait until you’ve got some ActionWebService code in place before hacking up your installation, feel free to come back to this section later. In any case, rest assured that the following steps will get you going with ActionWebService in no time. Note that this procedure assumes you have previously frozen your Rails distribution in vendor/rails by running:

Finally, in order to make use of the test scaffolding, which allows you to invoke XML-RPC requests using your web browser, you’ll need to patch the file ActionWebService/lib/action_web_service/scaffolding.rb under vendor/rails. Change line 114 from:

Creating an Abstraction Barrier

In the first half of this book, we concentrated on third normal form (3NF) and domain-key normal form (DK/NF). In this chapter, we’ll move away from this normalization restriction when we create an object model. That may sound surprising, given the emphasis placed on normalization at the data layer, but as we’ll see, normalization won’t help us here.

Before we proceed with creating an object model, we first need to understand what one is, and also what it’s not. Therefore, we need to take another look at ActiveRecord, from the perspective of the application as a whole. Once we’ve done that, we’ll see how an object model fits into the picture.

ActiveRecord As the Physical Model Layer

Think back to earlier chapters in this book and recall your thoughts as a simple schema turned into what at first glance may have seemed like an overly complex one. In many ways, that more intricate data model was easier to deal with because it removed the possibility of recording or producing incoherent data, which itself is no picnic to deal with.

Often developers shy away from highly normalized schemas because the abundance of tables and relationships seems too distant from the end result they hope to display on a web page. The data presented to users on web pages is usually not normalized, and if you work from display to schema, it will seem onerous (or even pointless) to obsessively normalize your data.

Such a perspective seems hard to argue against, what’s missing is a key element of data-driven website design. That element is the object model: the layer of abstraction above the physical layer. Its job is to make web pages easy to display. It was never claimed by anyone, anywhere, that a physical model’s purpose is to make web pages easy to display. The schema of the physical model maintains data integrity in your database, period. It’s the object, which has yet to be fully explained, that model maintains sanity—your sanity—in your presentation layer.

If you are born and bred on Ruby on Rails, you may be wondering why you have not heard of this distinction before. The reason you have not read about the difference between physical models and logical models in Rails is that Rails does not distinguish between them.

In Rails, the mantra is “convention over configuration,” and most tutorials are focused on showing the quickest way rather than the best or most scalable or maintainable way. Part of this ethos is an insistence that “magic” is good, especially at the data layer, where in Rails there is next to zero configuration. Since you don’t have a step in which you declare the nitty-gritty of your data model, the hope is that you can get started writing object model code right away in ActiveRecord models. That is the promise. But here are those words again: object model. Now we’re saying that ActiveRecord isn’t for object model code, but for physical model code. So what does belong in ActiveRecord models, if not object model code? What are they for?

In fact, there is quite a bit of “configuration” required in Rails on top of your data model. First, you do still need to tell Rails which tables exist by creating model classes based on ActiveRecord::Base for each table or view. You do need to define what relationships exist between those tables with has_one, has_many, belongs_to, and has_and_belongs_to_many. You also need to define validation for data-passing through ActiveRecord model classes to prevent invalid data from being saved: e.g., validates_uniqueness_of, validates_presence_of, validates_numericality_of.

Once you have put all of this information into your ActiveRecord model classes, what you have, in essence, is a directory full of database configuration. Convention saves you the necessity of declaring class names and foreign key column names. Of course, you are free to specify them, and you need to if you must stray from convention or are working with a legacy schema. But no matter how you look at it, the reality is that you have written quite a bit of configuration, it just was written in Ruby rather than in XML files.

The normal Rails convention would be to place application logic into ActiveRecord models, but this blurs—in fact, it removes—the line between the physical and logical models. This gets to the heart of why inexperienced database schema designers are uncomfortable with 3NF and DK/NF when viewed from the application layer, where they are most at home. They ask, “I have to design my UI with this?” Thankfully, the answer is no.

Before we move on to the object model layer, let’s first review our data model as we left it in Chapter 8. Figure 16-1 shows the schema, which has a one-to-one relationship with our ActiveRecord model classes. Review it, as soon we will be layering on top of it.

Figure 16-1. A review of our DK/NF movie showtimes schema

The Object Model Layer

Between the data layer (represented by ActiveRecord models) and the display layer, there is an object model layer that represents the problem the way it is natural to think about it. An object model need not be in 3NF or DKNF. As mentioned, the physical layer’s purpose is to maintain order at the data layer, whereas the object model’s purpose—based upon the assumption that the physical layer is doing its job—is to maintain sanity at the application layer. In other words, it maintains sanity for the application developer.

The object model for the movies portion of our schema might look like Figure 16-2. It is vastly simpler than our physical model, but it still contains all of the necessary information to:

Display current movies

Display details about a particular movie

Display details about a particular theatre

Display movies in a given location

Place an order

Figure 16-2. Movie tickets object model

Note the many things that we don’t care about here:

The rating information is repeated in every movie object

There is nothing to constrain the auditorium of a showtime to any set of actual auditoriums in a theatre

There are no “domain” objects to constrain values, such as ratings or zip codes

There are no foreign keys, but rather object-oriented “container” relationships

But at this layer, we don’t need to worry about any of this, because the physical ActiveRecord layer is worrying about it for us.

Think of it this way. First, we created a solid data model, keeping in mind that the database is a true piece of the application whole. At the physical layer we were concerned with translating this rock-solid layer into ActiveRecord models that would give us easy access to our data from Ruby. The next step is to translate this ActiveRecord layer objects into something that is easily accessible for passing through the service and for building UIs.

When building an XML-RPC service in Rails, we create object model classes based on ActionWebService::Struct. These classes translate data from ActiveRecord models into something that can be returned through our service API. Because Ruby is not a strongly typed language, and many consumers of XML-RPC services are, in ActionWebService::Struct classes, you need to define the types for all data members. You do this with the member keyword:

member name, :type

The allowable types are:

int

string

base64

bool

float

time

date

datetime

And of course, another ActionWebService::Struct class is an acceptable type as well.

For now, let’s focus on the classes related to displaying movies and showtime information, and for the moment put orders aside. We can define our classes as in Example 16-3.

Example 16-3. Class declarations for the MoviesService object model, lib/movies_service.rb, or in plugin

Here we have declarations of our classes, but no definition of how they work. What we have in Example 16-3 is very much like a header file in a C++ program. We’ve defined the structure of our data, but not how it works. This “header file” will have a special location in our architecture, but for now, if you want to get started following along, place it in lib/ so that Rails will automatically load it. We’ll move it later, when we come back to the header file analogy in our discussion of the service plugin. But first, let’s add some logic to the Movie class so that we can instantiate one based on data fro ActiveRecord objects. Example 16-4 demonstrates.

This file, movie.rb, gets placed in a new location in our application under the models directory. Notice that we have placed the class in the module Logical. Just as our physical models were in a module called Physical, and went in the directory app/models/physical/, this class and other class definitions for our object model classes will go in app/models/logical/.

You may have noticed that we just defined the class Logical::Movie twice: once in the file lib/movies_service.rb, and again in logical/movie.rb. This is not a problem, because in Ruby you can re-open classes as many times as you like to add more methods or data. However, while this is fine as a Ruby practice, re-opening a model class in this way does create a problem within the Rails framework. When your code looks for the Logical::Movie class the first time, it will have already been defined from within the movies_service.rb file. That prevents the Rails auto-loader from looking for the file in models/logical/. Rails has already loaded our declarations in the “header” file, but to make Rails do what we expect and load the definitions as well, we need to explicitly instruct Rails to load the logical model class definitions. We do that by adding the following lines at the bottom of in application.rb, similar to the method we used to make Rails play nice with our multiple table inheritance mechanism in Chapter 10:

You may be wondering why we don’t just put the declaration and definition of each class in the same file. Hang on to that thought, as this structure will make sense when we arrive at creating the service plugin.

Before we do that, since we have written a new type of class, let’s do the right thing and test it. Testing logical models is no different than testing physical models, which we’ve already done quite a bit of. Example 16-5 shows a simple test that retrieves a Logical::Movie object and checks through a series of asserts that the logical model matches the corresponding physical models.

The definition of our other object model classes proceeds similarly. Example 16-6 shows the definition of the Logical::Theatre class. Notice how on the way from physical to logical model, we moved the address information from the class itself into a substructure.

Figure 16-3 shows how the physical and logical models fit together. At the very bottom is the database, with a 3NF or DK/NF schema. Attached to the database are ActiveRecord models. All SQL queries occur at this layer and this layer only. The ActiveRecord models have two clients. On the right, ActionController classes intended for internal administrative use access them. On the left, the ActionWebService stack accesses the ActiveRecord models. First there is a translation layer from physical to logical model with the ActionWebService::Struct classes. The external-facing API deals in these logical object models. Neither the ActionController classes nor the ActionWebService classes ever access the database directly; they always go through the ActiveRecord layer. Each layer only sees the data directly below it. Now that we have a logical model layer, the next step is to define the ActionWebService API.

Figure 16-3. The pieces of a Rails XML-RPC service application

Defining the API

There are four components to a Rails service. We’ve already defined part of our logical model layer—the Movie, Theatre, and Showtime ActionWebService::Struct classes. Now let’s look at the other three pieces so we can begin making actual service requests.

The first is the API, which defines which methods are available to clients. Your API files go in the directory models/apis. You can split up segments of your API into multiple API files, just as you can split up different actions into multiple ActionController controllers. For now we’ll define a single, simple API called MoviesApi. Our API declaration is shown in Example 16-7. This API declares two methods, get_movie and get_theatre, which allow for retrieval of a Movie or Theatre object by ID.

First, we define each API method with the method api_method. It has three parameters. The first parameter is a symbol defining the method name. The second is an array defining the parameters the method expects. The third is an array defining the return value. The method name is required, but the expects and returns parameters can be omitted if the method takes no parameters, has no result, or both.

Note that each element of the expects and returns arrays is a one-item hash. Although this is not required—you are free to list only the types—this syntax is recommended, as it is self-documenting.

The third thing to notice in this code is that when declaring one of our logical model types as a parameter or return value, we need to provide the module name as well as the class name. This is because the API class is in global scope.

The second component of our XML-RPC service is the API implementation, which actually assigns code to each API method. These implementation files, which are based on ActionWebService::Base, are analogous to regular ActionController controller files. By default they should be placed in the app/models/services directory. However, functionally these classes are controllers, so we instead can put them in a more sensible directory by modifying the Rails load path. We already did this in Example 16-2 when we added app/controllers/services to the load path list.

Example 16-8 shows our API definitions for MoviesApi. Note that we must declare which API we are implementing using the web_service_api method.

The third and final component of putting together our XML-RPC service is defining a controller that passes service requests through to the ActionWebService. The endpoint URL is still just a URL to Rails, so we need a proxy class that translates requests destined for an ActionController class into those that can be handled by our service. Example 16-9 shows how we define such a class, which is placed in the standard app/controllers directory.

The first line, a call to web_service_dispatching_mode, defines the type of dispatching. In ActionWebService, three types are supported: :direct, :delegated, and :layered. We write our Rails code identically for delegated and layered modes, but for direct mode, we define methods directly in ActionController controller classes, not in a separate implementation model as in Example 16-8. The second difference is in the number of endpoint URLs. For direct and delegated modes, we have a different endpoint URL for each controller or API model class we define. In layered mode, there’s a single endpoing URL, and the API intended to be accessed is specified by the client in a protocol specific way. For XML-RPC, that’s done by prefixing the method name with the API name and a period. Figure 16-4 summarizes these differences.

Figure 16-4. Summary of ActionWebService dispatch modes

In practice, the differences between the modes is not significant. In this book, we’ll use layered mode. Having a single endpoint URL makes life easier for non-Rails clients, and separating some logic between controllers and models can give us some additional flexibility.

The second line contains a call to web_service_scaffold. Like ActionController scaffolding (which should never be used outside of testing), this generates a series of web pages where you can test your web service via a web browser. The parameter defines the URL where the scaffolding will be found. In this case, it would be found at http://localhost/movies_service/invoke.

The third line actually does the work of attaching an API model class to our service. You can segment your API into as many logical chunks as you want, and attach them all to the service one by one here.

With this much in place, you can now go check out the WSDL file that is automatically generated by Rails for SOAP clients. It’s also good documentation for clients using XML-RPC. The WSDL is located at http://localhost/movies_service/wsdl.

We’re also ready to write our first functional test to check that our API method, get_movie, works as expected. Example 16-10 shows how to write a functional test for a service method. Note that to make a service call, you use invoke_direct, invoke_layered, or invoke_delegated. We also need to require test_invoke.rb, which contains the unit test definitions for these methods.

More Testing

Aside from Rails functional testing, we can test our service application in other ways as well. The first and easiest is to use ActionWebService scaffolding. This is helpful when we want to see what service request results look like on the fly. Recall in Example 16-9 we declared a scaffold at invoke using the web_service_scaffold method. This defined a set of test web pages accessible at http://localhost/movies_service/invoke. Figure 16-5 shows the sequence of selecting a service method to invoke, setting up parameters for invocation, and receiving the response.

Figure 16-5. Testing an ActionWebService service with scaffolding

Note that if our service is facing the public Internet, we may want to turn this scaffolding off for production use. If that is the case, we can also test the service via a desktop client. On the Mac, a free XML-RPC client is available at http://ditchnet.org/xmlrpc/. To test a service, we define the endpoint URL, the method name, and the parameters. Figure 16-6 shows the sequence of using the desktop client to test a layered service.

Regardless of how we test, the XML generated for requests and responses is the same. The beauty of XML-RPC is that we don’t need to worry about what this looks like. However, for reference, a sample request is shown in Example 16-11, and a sample response in Example 16-12.

The Client Plugin

Accessing an XML-RPC service from another Rails application—presumably a thin front-end—is, like all things Rails, extremely easy. If we’re within a controller, such as ApplicationController, the following line is all we need:

In both cases, we then call methods on the local variable movies. In the first example, the first parameter to web_client_api defines both the local variable name of the client to be created, as well as which API file to look for, in this case, MoviesApi. In the second example, each of these is explicit.

Hopefully this seems strange. Why does the client need access to the MoviesApi class defined on the server? Such a constraint doesn’t seem like it provides the loose coupling we are after with a service-oriented architecture.

In reality, we don’t need to share any files between the client and server, but doing so is what allows the Rails client configuration to be so simple. In cases where we have control over both the client and server, and we are building both with Rails, it’s not hard to share the API definition class. For non-Rails clients, the API definition class is clearly not needed, and they can connect in their own language or framework specific way.

In order to share code between our two applications, server and client, we’ll build a plugin that for each installs in vendor/plugins.

The client plugin serves a few purposes. As noted above, it allows us to easily share code between our service server and clients. If we have multiple service clients, the plugin also ensures that they have a consistent interface to the service. We’ll create a wrapper around the simple client instantiation from above, which is described below as the client singleton. This wrapper, like any wrapper class, gives us the flexibility to take certain actions before, after, or around any service method invocation, on a per-method basis, or for every method. Having a shared client that we define once also lets us test the client apart from other code, and it facilitates integration testing.

Shared Code

The first steps to writing the client plugin are to move the code needed on both the server and client side into the plugin, and ensure the service itself still works as expected. Our unit tests will help ensure that nothing breaks as code is moved around.

Two pieces of code get moved into the plugin. First, the API declaration file movies_api.rb gets put in the plugin lib directory.

Next, the “header” file we created in Example 16-3 gets moved from the application-level lib directory to our plugin’s lib directory. Armed with the identical definitions of what data is in each logical model class via this header file, the server and client are free to re-open the classes locally and add methods appropriate to their role. On the service side, as we’ve already seen, the logical model classes in app/models/logical define, at the very least, how build the logical objects. Those classes can contain other business logic as well. On the client-side, we can re-open the classes in the same way and add convenience methods as we see fit.

We ensure our header class gets loaded, so the following line goes in the init.rb of the plugin:

require 'movies_service'

We don’t need to explicitly require the API definition class, as it is found automatically by the auto-loader.

At this point, we should rerun our tests to ensure everything is working. Visit the scaffolding test web page again and kick the tires.

The Client Singleton

The client singleton is a class that automatically creates a client for our service that is accessible from anywhere within the application, controllers, and model classes alike. It also ensures that access from each type of class is consistent, as we would expect it to be. Example 16-13 shows the shell of our client plugin.

The initialize method instantiates an ActionWebService client and assigns it to a protected instance variable @client. Rather than call methods on the client object directly, callers instead invoke the service methods directly against a service client instance. We have redefined method_missing at the instance level to forward these requests to the @client object. This allows for an opportunity to modify the arguments going in, the result going out, or perform other activities such as logging.

This class is a singleton, which ensures there is only one instance of the class in our application at any given time. Singleton class instances are normally accessed via ClassName.instance—in this case, it would be MoviesServiceClient.instance—but we’ve redefined method_missing at the class level as well to forward requests through to the singleton instance.

Example 16-13. Code for a service client plugin, lib/movies_service_client.rb

Note that we have used two constants in Example 16-13 that haven’t been defined anywhere: ENDPOINT_URL and TIMEOUT_SECONDS. These are configuration parameters that could be different from one client to the next. For example, the endpoint URL could change from one environment to the next. Different clients may have different notions of what is an acceptable time to wait for a service response. In Rails 2.0, configuration such as this is placed in an initializer, in config/initializers. Example 16-14 shows how to define these configuration parameters.

Integration Testing

Integration testing is the notion of testing applications via a third-party tool, treating components like black boxes. Unlike unit testing, which has access to the application and is testing the public interfaces of the code itself, integration testing tests the public interface of an application: the service interface.

Integration testing is different from functional tests like the one in Example 16-10, too. An integration test actually makes a remote service call to a running service to perform the test; it doesn’t just simulate the request. In fact, integration tests should be run against a live setup of your entire application stack, including databases with real data in them. It should be the same environment QA uses for testing the application.

An integration test also allows us to test how multiple remote components work together. For example, if a request to one service is expected to make a request to another service, we can check in an integration test that everything that was supposed to happen did in fact happen.

Don’t be intimidated by the term “third-party tool.” The third-party tool can be yet another Rails application. In this case, the entire function of the application is only to run tests. To get started with our Rails integration test framework, we simply create a new rails application:

rails integration_test_framework

We ensure ActionWebService is set up correctly, as described in the first section of this chapter. Then we import our service client plugin into vendor/plugins, and create an appropriate initializer configuration file like the one in Example 16-14. To get started, we must also turn off the components of Rails we aren’t using, so we don’t have to set up, for example, our database.yml file. In environment.rb, we uncomment this line:

In integration testing, we can be black-box regarding the service application, and white-box regarding that service’s database schema. We can connect directly to the database to insert records, and ensure that the service returns them in the expected way. This sort of testing is shown in Example 16-16.

Note that in this example, we defined just enough of our schema to get the data under test into the service. We defined the database connection directly in the test, rather than via database.yml, because our integration testing may one day span multiple databases, one for each service under test. Since this information is repeated, and also likely to change, it can and should also be moved out into an initializer.

Found this book useful?

I’m riding a bike 545 miles in June 2017 from San Francisco to Los Angeles to support HIV/AIDS-related charities. I would love to see readers of my book support this cause, too. If you got some use out of Enterprise Rails, please support my 545-mile charity bicycle ride taking place in June 2017 to say Thanks.