Introduction

Web services have provided exciting business opportunities for many individuals and enterprises. With fierce competition and shrinking margins, the Internet is not an easy place to conduct business. In addition, customer demands on greater flexibility and complex logic support are ever increasing. With this “big picture” in mind, if you want to maximize revenue from your Web Services business, your services need to be associated with an appealing suite of offers and pricing models. This article presents a technical implementation of several business models using the Microsoft .NET Framework that can be offered to attract Web Services customers. Models include providing a free trial of the Web Service, charging the customer by the number of calls, and charging the customer a monthly subscription fee.

Background

The concept of Web Services is about dynamic interactions between enterprises, partners, and affiliates, reducing the boundaries that are tightly coupled to technical implementation details between them. There’s no doubt that technology is a key enabler to make this concept an industry success, but it’s important to remember that Web Services serve as the foundation for the next generation of Internet applications. Taking advantage of the opportunities that Web Services offer is, therefore, a strategic business decision.

One question that commonly arises in this context regards the possible business models that Web Services vendors could offer to potential customers. When you look around, you will find different sites offering different usage models that suite their business processes and market expectations. Google, for example, provides limited (1,000 queries per day) free use of its spell-checker Web Service that anyone can use in an application. A large number of companies provide use of their Web Services for free, but usually, it’s in conjunction with some of their other online or offline services. Amazon.com, for instance, provides free access to its e-commerce services, but charges a fee to actually ship products. eBay, on the other hand, keeps track of the overall usage of Web Services and charges a fee based upon the access.

In this article, I present several ways to attract potential consumers with a variety of packages. Once you are ready to deploy your service to the general public, you need to address some fundamental issues in terms of enforcing financial rules, for instance:

What is the mechanism to identify the consumer of your Web Service?

What permissions does the client have for calling web methods?

How is the confidentiality of data that is sent over the Internet maintained?

These questions are basically about the infrastructure components that are associated with a typical commercial Web Service. You’ll learn how to build components that can help you enforce the rules defined in the packages you offer.

In this article, I focus on the technical side of business models that can be associated with a commercial Web Service. I use SOAP Extensions to create reusable components to manage authorized users of Web Services and to track Web Services usage by those users. I will first introduce the business models that I’m going to implement in this article and the core components that are needed to offer these models. Also, I’ll give some additional business model ideas that you can offer by extending the models implemented in this article. An overview of SOAP Extensions and their role in Web Services message processing is also provided. I’ll then explain the solution architecture, database schema, and key parts of the core components and the associated code. It’s important to note that the sample code provided with this article doesn’t touch on the security aspects of a Web Service; however, some useful related links are provided at the end of the article.

Using the Code

The sample Web Service provided with the article exposes three primary operations.

Provides the temperature of an area by zip code

Provides the latest available stock quote by its symbol

Provides the latest currency exchange rate

For brevity, I hard-coded the value returned by these web methods, but for actual use, these should be retrieved from some real-life application/database. The sample code included with this article basically comes in the form of several components (each with its own solution), as follows:

Two SOAP Extensions, one for user validation and the other for Web Services usage accounting.

Two ASP.NET pages, one for new user registration and the other for new service subscription.

A common DBManager component that is used by the SOAP Extensions and ASPX pages for database interaction and common data-structures.

A Web Service exposing the web methods as described earlier.

A Windows Form–based client application calling the web methods.

Detailed instructions about installing the sample are provided in the ReadMe.txt file included with the sample download.

Web Services: The Business Context

XML Web Services bring with them an exciting business model by delivering a piece of software as a service. With Web Services, you have the benefit of building business applications by integrating the best-of-breed components from different customers and vendors. It thus becomes much easier to host applications on the Web and reap revenues on a usage basis. It’s important to remember that such distribution applications need careful design, especially in the areas of handling scenarios where one of the services goes down or the mechanism of tracing errors is across distributed components spread over multiple heterogeneous systems, etc.

Before you can start marketing your Web Services, you’ll need to formalize the usage-charging mechanism. Depending upon your target audience, the service offering may vary. The following table has three generic charging schemes that we’ll examine in this article.

Service Offering

Service Description

Rates

Standard

The customer buys a fixed number of calls to a service. Each call is tracked by the system. Once a customer runs out of purchased calls, usage of the service is blocked.

1,000 calls for $1.20

Premium

The customer pays a monthly fee to make an unlimited number of calls to a service per month. The system keeps track of the time when the customer subscribes to the service, and if the customer decides to discontinue the subscription, usage is blocked.

$35 per month

Promo

This offering is basically an attempt to boost the business during the hours of day when it’s not attracting a lot of customers. It’s much like the same as the Standard offering, where the customer buys a fixed number of calls, with the addition that the customer can make the calls only during preconfigured hours of the day.

3,000 calls for $0.90, only between the hours of 18:00 and 23:00

A generally recommended practice is to offer a free trial version of your Web Service. This basically lets potential buyers test-drive your application to see if it meets their needs. The service is typically very limited, though, under trial mode. For the purposes of this article, a customer will be able to use the service under trial mode for three days, with at most a maximum of 1,000 calls per method.

For the purposes of this article, I implemented the three models shown in the preceding table, but you can easily enhance these to suit your own requirements. For instance, once a user’s subscription expires, instead of blocking any more calls from the user, you could move the user to the next pricing structure. In addition, a real-life service agreement needs to include details on the response time and availability of the Web Service. For instance, the Premium service is offered at $35 per month for a guaranteed 10-second response time, with 95% system availability. In addition, depending upon the customer base, there could be some other flavors of these offerings based upon the rates at which the calls are serviced and providing historical data (e.g., out-of-date stock quotes), etc.

The Solution

The following core components are required to meet the business objectives and design goals just laid out:

Registration management: The ability to uniquely identify a customer.

Subscription management: The ability to provide the customer access to various services under agreed-upon terms and conditions.

Authentication: Web Services must be secured in order to restrict the access of Web Services methods to registered users only.

Usage accounting: A usage tracking system is needed to monitor the usage of Web Services methods.

Registration and subscription management components can easily be implemented using ASP.NET. I will show you these later in this article. You can essentially take two approaches to implementing authentication, usage accounting, and message encryption. The first is to make them part of the Web Service itself. So, for example, for authenticating a web method call, you would pass the user information (user name, password, and billing information such as address, credit card number, expiration date, and so forth) in each method call. The other approach is to implement them as reusable components that can be used with any Web Service. Some of the advantageous of the latter approach are as follows:

Developers of a Web Service can focus on the business functionality without worrying about the rest of the components.

If business rules change for these infrastructure components, there is no need to make any changes in the actual Web Service code.

These components could be used across several Web Services, resulting in lower development and QA time and efforts.

As mentioned previously, SOAP Extensions can be used to implement such reusable components for user validation and Web Service usage metering. The user information can be passed via SOAP headers, which can then be easily read during the appropriate stage of a SOAP message’s processing. The Microsoft .NET Framework provides some powerful features that can be used to implement these types of components. To leverage these features, let’s first try to understand precisely what happens on the web server when a client makes a request for a Web Service.

Web Services Message Processing

The whole process begins with a client making a request to the web server (IIS, in our case) hosting the Web Service. IIS passes the incoming requests to an ISAPI filter, which routes them to an ASP.NET worker process. The worker process basically hosts a CLR instance, and is responsible for providing an application domain for each virtual application configured under IIS. Within the application domain, the request passes through specified HTTP modules, and then to a specific HTTP handler factory. HTTP modules can view or change the contents of a request or response message as they pass through the pipeline. In addition to HTTP modules, requests to Web Services also pass through a number of configured SOAP Extensions. The figure below shows the flow of an incoming request through a set of HTTP modules and SOAP Extensions before actually invoking the web method:

Web Service message processing

Before I go into a detailed description of SOAP Extensions, it’s worth mentioning the difference between SOAP extensions and HTTP modules. Both provide the opportunity to process a requested message before the web method is invoked, and to further process the response before it’s transmitted back to the client. It’s important to remember that each acts at a different stage during a message’s life cycle, and therefore, each has a different level of applicability. For example, HTTP modules provide access to the HTTP request and response in its entirety, not just the SOAP message. Also, HTTP modules are specific to the web application that contains them, not to the Web Service itself. SOAP Extensions, on the other hand, have access only to a SOAP message, and therefore are more suitable for implementing tasks specific to a Web Service or a web method. It’s noteworthy that SOAP Extensions do have access to the XML contents of a stream just like HTTP modules. Furthermore, SOAP Extensions can be set up to process each and every request and response, or they can be applied declaratively to individual web methods. SOAP Extensions also offer flexibility in the sense that they can be deployed on the server as well as the client side.

SOAP Extensions

SOAP Extensions provide a way to plug code into the Web Services infrastructure. They allow you to inspect or modify SOAP messages at specific stages in the message processing before they are transmitted over the wires. Their ability to process SOAP messages outside of the actual Web Service implementation makes them a good candidate for implementation of a reusable infrastructure component.

When a Web Service client invokes a web method, it converts the objects and parameters involved into an XML-based SOAP request through a process called serialization. The reverse process, called de-serialization, happens on the server side to convert the XML stream back into objects. When the server sends back a SOAP response to the client, it goes through a similar process of serialization and de-serialization. The figure below depicts the role of SOAP Extensions in the Web Services architecture:

Key events invoked during a SOAP extension’s processing

The small solid circles in the above figure show the points in the serialization/de-serialization process where the incoming/outgoing messages can be tapped in SOAP Extensions. The following table provides more details of these stages:

Message Direction

Stage

When the Stage Occurs

Incoming messages

BeforeDeserialize

Just before the SOAP message is de-serialized into an object

AfterDeserialize

After the SOAP message has been de-serialized back into an object

Outgoing messages

BeforeSerialize

Before an object is serialized into a SOAP message

AfterSerialize

After an object has been serialized into a SOAP message

SOAP Extensions Creation

SOAP Extensions are classes that derive from the abstract class SoapExtension, which lives under the System.Web.Services.Protocols namespace. The following code block shows the key methods of this class:

The two SOAP Extensions implemented in this article are derived from this class. ProcessMessage() is the method of key importance, as this is where the incoming messages are intercepted and processed. I’ll cover this topic in more detail later in this article.

SOAP Extension Lifetime

One of the GetInitializer() methods can be teamed up with the Initialize() method to do any initializations that are needed during the Web Service’s lifetime. GetInitialize() has two overloaded methods, one of which is invoked during the lifetime of the application depending upon how the SOAP Extension is configured. I will elaborate on the significance of these two methods a little later, but at this point, it’s important to remember that GetInitialize() is invoked only once per HttpApplication instance. It returns an Initializer object that is cached by ASP.NET and passed to the Initialize() method. Initialize() is invoked for every request, and it’s the job of this method to create a new instance of the extension for each request.

ChainStream() and ProcessMessage() are then invoked for each request before and after the web method invocation. ProcessMessage() is the method of key interest here. When overridden in the derived class, this method allows you to receive a SoapMessage instance that gives access to all the request/response information such as SOAP headers, message streams, and even information about the web method. The ProcessMessage() method is executed four times during each Web Service method invocation, once for each of the four stages tabulated in the preceding table. Each time ProcessMessage() is called, the current request or response processing stage can be queried by the Stage property of the SoapMessage parameter.

Sequence of methods called during a SOAP Extension’s lifetime

Configuring a SOAP Extension

A SOAP extension can be configured in one of two ways depending upon how it’s intended to be used:

If a SOAP Extension has to process each request made to its web methods in a Web Service, then an entry for the extension should be added in the application’s web.config file, as follows:

The preceding SOAP Extension runs with all the Web Services that are within the scope of this configuration file. I also specified that the SOAP Extension be run within the relative priority group 0 and a priority of 1.

If the SOAP Extension has to process a request to a particular web method in a Web Service, then the extension needs to be associated with a custom attribute. You’ll have to then apply it to each XML Web Service method that you want the SOAP extension to run with. A customerattribute can be created by inheriting a class from SoapExtensionAttribute and overriding its two properties, Priority and ExtensionType. The Priority property is used to get and set the priority of the extension, whereas ExtensionType is a read-only property that returns the class type of the extension that is attached to the attribute.

Chaining Multiple Extensions

Any number of extensions can be chained in the pipeline. The order of execution of chained extensions is of vital importance. The order in which the ASP.NET runtime invokes a series of extensions is dictated by its priority attribute—the lower the value of this attribute, the earlier that extension will be invoked to process the incoming request.

Error Handling with SOAP Extensions

Web Services send exceptions back to the clients in the form of SOAP faults. A SOAP fault consists of a faultcode, faultstring, sometimes a faultactor, and lastly the detail about the source of the exception. From a programmer’s perspective, though, throwing a SOAP fault is just like throwing an exception from any other .NET application. The ASP.NET framework takes care of converting the exception into a valid SOAP fault element in the SOAP message. If you are using a .NET client, you can catch a SOAP fault by setting up a simple try/catch block. The downside of SOAP faults is that they are all of the same type—SoapExtension—and therefore, it’s not easy for a client to determine what exactly went wrong.

Architecture of SOAP Extensions

SOAP Extensions offer a rich and extensible mechanism for implementing reusable infrastructure components for Web Services. Their ease of implementation without losing flexibility has made them the “way to go” for the developer community. SOAP Extensions–based reusable components will be the backbone of Web Service offerings, and enforces business rules without distracting from the design goals defined at the start of this article. The following list provides a few broad categories where these types of components can be used:

Data encryption

Authentication and authorization

Accounting/logging

Monitoring system performance

The figure below depicts an example of this process flow:

High-level architecture diagram

I’ll also implement the registration and subscription management components that I’ll detail later in this article. The figure below shows the UML class diagram, ignoring the function signatures and a few other details that we don’t care too much about at this stage:

Class diagram of the proposed solution

As depicted in the data flow diagram shown in the above figure, the actual Web Service layer remains a small part of the application, with authentication, authorization, and Web Service usage accounting mechanisms all implemented as SOAP Extensions. SOAP Extensions are created to examine the SOAP header and get the user registration identifier to make sure the user has valid credentials. Web method information is available in an extension that can be combined with this registration identifier to determine if the user has a valid subscription to the method requested. If the user is not authenticated, a SOAP fault will be sent back to the client. In the case where the user has valid credentials as well as a subscription, the request will pass on to the web method. This mechanism frees the web method from ensuring user validity. Once a web method is invoked, a usage entry will be recorded in the database.

For usage accounting, I do not want to block execution within ProcessMessage() until the web method usage activity is recorded in the database. A better way of doing this is to queue this call in a thread pool. I discuss this process in more detail shortly during the discussion of the respective component. Also, it is very important to turn off the HTTP POST and GET protocols so that SOAP is the only supported protocol for these services. By disabling these protocols, you basically ensure tighter security, and mitigate the risk of external attacks. You can disable these protocols in the web.config file, as shown here:

With the theory behind SOAP Extension creation, configuration, and chaining behind us, let’s now move on to the implementation of two SOAP extensions, one for user validation, and the other for usage accounting. The UML sequence diagram shown in the figure below depicts the call flow within the various classes of the application:

Sequence diagram of the proposed solution

Database Design

All the key information related to the services offered, users, and services configuration is stored in the database. The following figure shows the database schema for the components used in this article’s proposed solution:

Database schema of the proposed solution

Let’s walk through each table and its purpose in the sections that follow.

WSDM_Services

This table contains all the web method names that our service is offering. So, for this article, there are three entries in this table, as shown in the following figure:

Services table

WSDM_Configuration

This table holds all the configuration parameters—for instance, the maximum number of days and the number of calls allowed during the service’s trial period, the hours of day for services under the Promo offering, etc.

Configuration table

WSDM_SubscriptionType

This table contains the definition of the subscription types and the subtypes that a user can subscribe to.

Subscription type table

WSDM_Users

This table contains information related to the users of the Web Service (first and last name, e-mail address, and registration key). This table is populated when a user registers by submitting a registration form provided with the Web Service support page. In a real-life application, there will also be some information related to the user, such as payment method, billing address, etc.

WSDM_UserSubscription

This table contains information about the user and the services the user subscribed to (e.g., service name, subscription type, and subtype). The Active column in this table is used to indicate the currently active and expired subscriptions.

WSDM_ServiceUsageTracking

This table contains tracking information, such as the date/time a user invoked a web method the user subscribed to.

Note: The first three tables just described are somewhat static for the purposes of this article. I have provided a Stored Procedure to populate these tables in the sample code. You can find more details on this in the ReadMe.txt file included with the sample code.

Components Description

In this section, we’ll walk through the details of all the components within the system.

Registration

Registration is the process of establishing the identity of the consumer of your Web Service methods. The figure below shows the web page that a user will use to sign up in order to get an identity for consuming the Web Services.

Registration page

Once a user has registered using the web page shown in the above figure, the user will be assigned a registration key that will be e-mailed to the address provided during the registration process. The user will have to use this registration key to get authenticated and use the Web Service methods successfully. Again, in a real-life application, a user will need to provide more information (such as billing information, etc.) during the registration process. The following Stored Procedure is used to save the user in the database:

I have used the Transact-SQL (T-SQL) NewID() function to generate the registration key for the user. The NewID() function basically creates a globally unique identifier (GUID). Once the user registration is successful, an e-mail will be sent to the user’s e-mail address. Please note that I have used the SQL Server xp_sendmail extended Stored Procedure to send the registration mail. You will have to set up this Stored Procedure in your environment to send an e-mail notification. If you do not have this Stored Procedure configured in your environment, you will get an error about xp_sendmail during the registration process, but you should still be able to finish the rest of the tasks. All you have to do is query the WSDM_Users table to retrieve the registration key.

Subscription

Subscription is the process of designating a user as an authorized consumer of a Web Service method. Once a user is registered, the user can use the subscription page as shown in the figure below to select the service and package of his or her choice:

Subscription page

Although it’s not shown here, a production application would require additional information, such as detailed terms and conditions for the user to accept, payment method for the setup fee, etc.

User Validation

There are two aspects of validating a user. The first is to ensure that the user is a legitimate user, and the second is to verify that the user has a valid subscription to the requested method. I developed a simple SOAP Extension to handle these two aspects. This code looks for a SOAP header of type ClientMessageHeader in the SOAP envelope. This type is declared in DBManager, and encapsulates user registration information. I put the code in the AfterDeserialize() handler. As mentioned previously, the AfterDeserialize stage occurs after an incoming request containing a SOAP message for invocating an XML Web Service method is de-serialized into an object, but prior to the actual method invocation. To identify the headers received as part of the SOAP message, I have to wait for the AfterDeserialize event to occur, since the incoming XML has not been parsed in the BeforeDeserialize event. At this point, the code gets the registration key from the header and validates it against the database. A SoapException is thrown if the key is not valid or if there is a server or database error. The following code block shows the section of code I use to validate a user:

For the purposes of this article, I didn’t go for a bulletproof method of validation. In a real-world application, this data should be encrypted and digitally signed to ensure authenticity and to prevent tampering. The purpose here was to demonstrate the use of SOAP Extensions to perform this task. You may want to modify or extend this code to implement more-sophisticated validation techniques to suit your requirements. There are some useful links provided at the end of this article where you can find more information about the security aspects of Web Services.

Let’s now go through a brief overview of how the usage of services is managed in different modes. If a user has subscribed to the service in trial mode, I first look into the configuration table to see the maximum number of days and calls that are allowed in trial mode. After that, I compute the number of days and calls the user has made so far. If any of these exceed the allowed limit, I designate this user as invalid. The method for validation of subscription in the full version is very similar to what I have for the trial version; the only difference is that the maximum allowed number of days or calls used from the subscription table is populated based on what service the user has subscribed to. For the non-peak-hour Promo service, I do an additional check that basically determines if the time when the user requested the service lies in the range configured for usage of service in this plan. If it is not, then the user is not allowed to access the service. Please note that all the times used in the system are GMT.

Usage Accounting

Usage accounting is implemented as another SOAP extension that can only be initiated if the user has successfully been validated. The following code block shows the code for the BeforeSerialize() handler.

As shown in the preceding code, I delegated the task of usage accounting to a ThreadPool object, as this task does not affect the normal processing of a message and I do not want ProcessMessage() to be blocked until this task is completed. The ThreadPool object is responsible for managing a pool of threads. This pool already has some threads created and waiting for work items. Once they finish executing an item, they wait for another one. For executing code in a thread pool, I simply have to provide a WaitCallback() delegate. This delegate does not return anything, and takes a single parameter of type object. Optionally, you can specify a parameter object to the function. A call to the QueueUserWorkItem() method of the ThreadPool class queues the UsageAccouting() method for execution. This method will be executed once a thread in the thread pool becomes available. The following code block shows the delegate function UsageAccouting():

One important thing to note is how I initialized the priority of the attributes of the UserValidation and UsageAccounting extensions. The code snippet for UserValidationExtensionAttribute and UsageAccoutingExtensionAttribute is where I initialize the values of Priority attributes.

By setting the priority attribute of UserValidationExtensionAttribute lower than that of UsageAccoutingExtensionAttribute, I ensure that the UserValidationExtension extension always gets a chance to process an incoming request before UsageAccountingExtension.

Subscription Processing

I still need a mechanism that can check for all the active subscriptions and mark any expired subscriptions. I do this by setting up a Microsoft SQL Server batch job. For the purposes of this article, this job is scheduled to run at the beginning of every day (00 hour, 00 minute), but for a real-life application, this job might need to run more frequently. The job basically calls a Stored Procedure that compares all the currently active subscriptions with their expiration limits. Depending on the subscription type (Standard, Premium, or Promo), the expiration limit can be based on either the number of days the service is used, or the number of calls made so far. Subscriptions for which the limit has been reached will be marked as inactive by setting the Active column in the WSDM_UserSubscription table to 0. In a real-life application, you should also notify the customer of this event by sending an e-mail asking the customer to resubscribe.

The following code block shows the code used to determine if for the active subscriptions that are subscribed for the full version, with either the Standard or Promo package, the number of days subscribed are expired. Once these subscriptions are singled out, I will update the WSDM_ServiceUsageTracking table, setting the Active column to 0.

Future Work

True interoperability is considered the heart of the Web Services world. To exchange platform-level information in a secure and reliable manner, Microsoft teamed up with other industry leaders to standardize the communication protocols and mechanism between Web Services. These specifications are collectively known as Web Service Enhancements (WSE), and are based on widely used standards like SOAP, XML schema, and WSDL. Web Services Security (WS-Security), for instance, leverages XML encryption and signatures to specify security solutions in SOAP messages. WSE basically provides a pipeline of input and output filters that can be used to inspect and modify SOAP headers/messages. You could use WSE custom filters to create a more elegant and extensible framework to address user authentication and web service metering issues.

Conclusion

Attaching attractive offers and deals to your Web Services is vital to succeed in this line of business. The Microsoft .NET Framework provides excellent support to enable these offerings with your Web Services via reusable SOAP Extensions. In this article, I presented a few basic but effective business models that you can offer to attract consumers to your Web Services. I then provided an overview of how Web Services process messages, highlighting the points of interest in creating this solution. Lastly, I broke down the requirements in components, and went through in some detail of how each component works. With the information presented in this article and the sample code, you are well on your way to being able to provide various pricing and usage models to your Web Services customers.

Points of Interest

The order in which the ASP.NET runtime invokes an extension when chaining multiple SOAP Extensions is dictated by their Priority attribute. The lower the value of this attribute, the earlier that extension will be invoked to process the incoming request. This can be critical when you have multiple extensions that are required to happen in a specific order.

Share

About the Author

Kamran Bilgrami is a seasoned software developer with background in designing mission critical applications for carrier grade telecom networks. More recently he is involved in design & development of real-time biometric based security solutions. His areas of interest include .NET, software security, mathematical modeling and patterns.