Introduction

Solution architect: We should start concentrating more on frameworks!!

Dumb developer: Well, I think I know about the .NET Framework. You mean, something like that?

After reading this article, you'll be able to:

Change your mindset a little bit, and start thinking about 'frameworks' instead of just 'code'

Understand a lot about practically applying the Provider pattern in your projects

Gain much knowledge regarding how XML configuration files and providers work (especially in the context of .NET)

1. The Problem

You are the Solution Architect of a big credit card company. You need to publish your credit card data to various banks in order to get it approved. Basically, you send the card information (say, as an XML file) to a bank, the bank may or may not approve each card request, and you need to get the result back. The format you need to publish your card data is different from bank to bank, and you need a separate interfacing standard for each bank. Some banks support only FTP, and some other banks support only Web Services - and so on. Basically:

You want to fetch your card data from a data source

Convert it to the bank specific format (like, an XML file or a CSV file)

Push it to a bank for processing

Synchronously or asynchronously pull it back/get the result

Save the result for each card in your database

You may do this process based on a scheduler - because you need to send card data to banks a few times a day (once or twice) and pull the results back.

And to make matters more interesting, more banks will be added to your system in future.

Well, now you need to propose a solution. Interesting.. huh?

2. The Solution

Let us see what all solutions are not good enough.

You should not think about developing your own scheduler - because you can use an existing scheduler to run your application to push/pull data to/from each bank.

You should not think about developing a separate application for synchronizing with each bank - because it is difficult to create, maintain, and deploy.

Let us see what all decisions are good design decisions.

You may develop an application to communicate with all banks. The application will load a list of plug-ins to push/pull data to/from banks from a configuration file.

Your application can be scheduled using Windows Scheduler the way you like.

So finally, let us decide to:

Build a 'Publisher' application, with some capability to load plug-ins and push/pull data.

Now, you speak with some of your friends and they ask you.. "Oh dude! Why can't you make it a generic interfacing Framework so that we can re-use it?". "Quite a good idea" you feel .. "But man, how exactly will I make it a Framework?" :|

There we are :). Let us see.

3. The Framework

Basically, a framework is:

Something you can re-use (give me the ready-made pack, I'll use it, or else, get away).

A set of libraries and support programs (like the .NET Framework).

A set of pre-solved problems to help you build solutions faster.

Solution architect 1: "The day we complete building frameworks for all human work flows happening in this world, solution architects may go out of job".

Solution architect 2: "No, we can still design and build frameworks to fix our existing frameworks".

Solution architect 3: "Dude, what about a framework to build frameworks? Probably a meta-meta framework or a meta-meta-meta framework."

Dumb programmer (thinks): "All solution architects are dumb, lazy, and crazy. They just draw diagrams and talk Greek. The whole project is in my head".

Let us come back to our topic. How do we create a framework for solving the above design issues? Here are some final thoughts regarding the framework we are going to build:

There should be a 'Publisher' framework to load plug-ins

The Publisher will call certain functions in each plug-in (say, Put and Get) to push the data and to pull the data

A simple front-end application will use the Publisher framework to load plug-ins from an XML file and to push/pull data

We may use the Windows Scheduler to schedule the front-end application.

Now, as you see, we need to load plug-ins after reading the information from an XML configuration file. Essentially, we need to separate some kind of functionality from the actual application, i.e., the publishing logic (push/pull data to/from bank) should be decoupled from the actual application (loader/scheduler) through a plug-in model so that we can add/remove and configure plug-ins at a later stage very easily.

Hence,

When a new bank comes in, we can develop a simple plug-in for data push/pull and deploy it easily.

Modify the configuration file so that the loader will load it.

Well, this may ring the bell. You suddenly think.. "separating functionality using a plug-in based model is nothing new - and an existing Design Pattern should be in existence for this". Well, that is what (actually, some what :-)) the Provider pattern is all about. Let us see how we can design our framework around the Provider pattern to solve the design issue.

If you are not so familiar with Design Patterns, you are lagging at least one step. Stop here, go and read my articles on 'Applying Design Patterns' - Part 1 and Part 2, then come back.

4. Designing the Framework

Altogether, the idea about the 'Provider pattern' is quite simple. There are various ways of implementation; here is a quite simple one:

You have a well defined plug-in interface - public interface IMyInterface, or even an abstract class, in your Framework project.

5.b. Creating the PublisherFactory

The publisher factory has a very simple static function, CreatePublisher.

The Publisher factory simply loads the type (i.e., our plug-in class) from the assembly, creates an object of it, and returns it. Looks so simple, huh? If the type we loaded is not an IPublisher, an error may occur at the line ipub=(IPublisher) pub because explicit casting may fail. This way, we make sure that all plug-ins we load should obey the IPublisher interface. It will also call the Init function to pass the parameters to the plug-in after loading it.

5.c. Creating the PublisherLoader Class

If you remember, one of our requirements was to load the plug-in information from an XML file. The PublisherLoader class is exactly for that. This class is not there in the above UML diagram - pardon me, but it is still a part of the framework. It will load an XML configuration file, load each plug-in specified, instantiate it by calling PublisherFactory, and call other functions in the plug-in like Get() and Put(). Before we go to the class definition, here is some more meat regarding how to create your own XML configuration files and how to actually read them.

The XML Configuration File

For example, consider that we should allow our framework to load plug-ins by reading a configuration file like this:

Each Publisher has various properties like AssemblyName and ClassName for specifying the plug-in assembly path (DLL) and the name of the plug-in class. Also, each Publisher has a set of settings, like Host and Port. These settings are passed to the plug-in class by the PublisherFactory class when we instantiate an object of the plug-in, by calling the Init function. How does this work? Read on. How do we read a configuration file like the one above? Fortunately, it is very simple in .NET. The steps are:

Create an XML schema to describe the configuration file format (i.e., to describe the elements and attributes in your configuration file).

Create a set of classes from the XML schema (to serialize and de-serialize an XML document specific to this schema).

Use these classes to read a configuration file like above to an object model.

If you see the above schema, there are three elements: Publishers, Publisher, and Setting. It is easy to understand, but you will see, the Publishers element can contain more than one Publisher element. A Publisher element has four attributes - Name, AssemblyName, ClassName, and Version - and a Publisher element can contain more than one Setting element.

A Setting element has two attributes- a Key and a Value. This schema exactly describes the configuration document structure we need. Save the schema to a file named Publisher.xsd.

Creating a Set of Classes

Now, to create a data structure out of our schema, I've several options. My favorite option is to use XsdObjectGenerator. It will generate a set of classes which are capable of representing my schema. I.e., it will create classes to create an object model of the schema, to which I can de-serialize my configuration file. Let us create the classes from the command line, using xsdobjectgen.

xsdobjectgen Publisher.xsd /n:Sumeru.Publisher.Framework.Data

This will create an object model based on elements in Publisher.xsd (i.e., a set of classes like Publishers, Publisher, Setting, etc., along with the attributes) to describe the elements in Publisher.xsd. The /n switch specifies the namespace for the newly created classes. And this creates a file Sumeru.Publisher.Framework.Data.cs with all the classes - please see the attached source code if required. There will be a Sumeru.Publisher.Framework.Data.Publishers class (if you remember, Publishers is the root element according to our schema), and we can use an object of this class to de-serialize an XML document.

Load the Configuration File

This part is pretty simple. We just create an object of the Publishers class and de serialize the XML file to it.

If you remember, the Publishers element has a collection of Publisher elements, and each Publisher element has a few attributes like class name and assembly name. We iterate the Publisher elements to pass the class name and assembly name to the PublisherFactory we wrote earlier. This will dynamically load a publisher, and may return us an IPublisher object - if the assembly name and class name is specified correctly in the XML file. Then, we may call our custom functions.

The PublisherLoader Class

The PublisherLoader class does all this work (though it is very simple):

Here, as you can see, we load the configuration, iterate each Publisher element, and pass the class name and assembly name to PublisherFactory for creating an IPublisher object. Then we call the functions Get() and Put(). If you notice, we are also passing the Setting collection of a Publisher element to PublisherFactory, and PublisherFactory will in-turn pass this Setting collection by calling the Init(..) function in a Publisher object during instantiation (see the code of PublisherFactory). We are almost done. Let us write a simple plug-in for our framework now.

6. Developing a Simple Plug-In

To develop a plug in, I've done the following steps:

Created a Class Library project named Sumeru.Publisher.FTPPublisher

Added a reference to our Framework project

Created a class named FTPPublisherPlugin by implementing the IPublisher interface in our framework

Here is the FTPPublisherPlugin class. It just implements all the functions defined in IPublisher. You may need to add your own code for putting and getting files to/from an FTP server, in place of showing message boxes.

7. An Application to Test the Framework

Now, let us develop a very simple Loader application which consumes our framework. It basically invokes the LoadPublisher function in the PublisherLoader class we created earlier. I created a form, with a text box which has the path to the XML configuration file.

The XML configuration file has information to load our newly written plug-in. I am just loading the same plug-in twice, with different settings. Try developing yet another plug-in, and add a new Publisher element to load that. Make sure that you are copying the DLL to the path of this loader app - if you are not explicitly specifying the full path for the DLL.

As you can see, we are using the PublisherLoader class to read the configuration file and to load the plug-ins, as explained earlier. Now, if you click the Load button, the plug-ins will be loaded, and you may see the message boxes from the plug-in we developed. (Let us keep it simple ;)

8. Design Variations

Here are some interesting design variations so that you can implement for practice, and for some additional brainstorming:

Yesterday, I spent some time with Abhilash and he was mentioning how he is planning to use a script engine in his provider-like framework. Instead of developing separate classes as plug-ins, there will be a core engine, and it will load the script files which can be interpreted by the core engine from a configuration file. That framework is mainly for data mapping between protocols.

Right now, the framework is not passing any data to the plug-ins (other than the settings). Sometimes (most of the time), this framework may need to pass data to/from plug-in functions (like, data to put and get). If the data passed from the framework to put/get functions are standardized (i.e., a pre-defined schema), everything is fine - but what if the data format is not pre-defined? How will you solve this issue? Post your comments here :)

9. Conclusion

Hope you enjoyed it. I tried to keep it very simple. Some more information if you are health conscious:

When you work with your comps, normally your entropy will go high (if you are a smart programmer), and this may decrease the level of water in your body. Make sure that you are drinking lots of water - this will eliminate a lot of stress, strain, and depression.

And finally, I'll also recommend an Art Of Living Part I workshop for you (see http://www.artofliving.org/ and have a look at the courses). It is an interactive workshop of 18 hours spread over 6 days. As it did for me, I hope that it may help you to find the right balance between your work and life - to improve the mental strength and clarity of your mind, and to improve the quality of your life. Here are some other popular articles I wrote:

I know what it takes to write a good article and I know it is easier to criticize. I highly appreciate your effort, also the guidance given here. I agreed that building framework is a good concept and will have its good share in future IT world.

There are lots of articles out there presenting very misleading/ wrong/ totally wrong concepts of software architecture. In that sense your article is OK, but since you are an architect and also lots of newbie with hunger to become architects read these article, I just thought to point out certain things which might help you to improve this a bit.

Article Says: You have a well defined plug-in interface - public interface IMyInterface , or even an abstract class, in your Framework project

It is a fundamental decision and is not at all, a one, “or” the other, selection in between interface vs abstract class. There are many things you can consider when choosing an interface over abstract class or wise versa. If you want your design to be interchangeable and want to hide the implementation and want to talk about a concept/ idea then choose an interface but if you want to define the base for a class and also want to force certain implementation but don’t want it to be instantiatable then choose a abstract class. There are many more to consider but I will write more on this with my next article..

You may implement an interface via a class but you cannot impliment an abstract class via another class, then you inherit it, and abstract class is of type class so you inherit it. In that sense the above statement is not correct..

Additionaly, you may also say that one interface inherit another interface.

Article Shows: According to the diagram..
FTPPublisher implement IPublisher, you show it as FTPPublisher inherit IPublisher, so the notation in between two has to be changed too..

There are many other small mistakes in your article, and also you have taken some fundamentally wrong decisions too, finally I am not sure whether you have really identify the concept of a framework correctly ..

Both this article and to some extent other 'arhitect' solutions like Fowler's one are simply inefficient, overly simplistic and quite frankly demonstrating the state of sad Java and CLR induced damage as well as OOP and imperative sickness induced.

If I ever saw an inefficient framework it is presented in this space.

Casting tells a lot.

All is a string and all is a setup tells more.

As for Fowler and his idea of framework vs library:

"A library is essentially a set of functions that you can call, these days usually organized into classes. Each call does some work and returns control to the client."

"A framework embodies some abstract design, with more behavior built in. In order to use it you need to insert your behavior into various places in the framework either by subclassing or by plugging in your own classes. The framework's code then calls your code at these points."

He should look into the libraries that are not even to date (after more than a decade) even remotely emulated in any of those garbage runtimes for mass public.