Introduction

Plugins are the common way for extending applications. They are usually implemented as DLLs. The host application locates the plugins (either by looking in a predefined folder, or by some sort of registry setting or configuration file) then loads them one by one with LoadLibrary. The plugins are then integrated into the host application which extends it with new functionality.

This article will show how to create a host EXE with multiple plugin DLLs. We'll see how to seamlessly expose any of the host's classes, functions and data as an API to the plugins. There will be some technical challenges that we are going to solve along the way.

We'll use a simple example. The host application host.exe is an image viewer. It implements a plugin framework for adding support for different image file formats (24-bit BMP and 24-bit TGA in this example). The plugins will be DLLs and will have extension .IMP (IMage Parser) to separate them from regular DLLs. Note however, that this article is about plugins, not about parsing images. The provided parsers are very basic and for demonstration purpose only.

There are many articles describing how to implement a simple plugin framework. See [1], [2] for examples. They usually focus on two approaches:

The plugin implements a standard (and usually small) set of functions. The host knows the names of the functions and can find the address using GetProcAddress. This doesn't scale well. As the number of functions grows the maintenance gets harder and harder. You can only do so much if you have to manually bind every function by name.

The function returned by GetProcAddress is used to pass an interface pointer to the plugin or to obtain an interface pointer from the plugin. The rest of the communications between the host and the plugin is done through that interface. Here's how you do it:

The interface way

Interfaces are base classes where all member functions are public and pure virtual, and there are no data members. For example:

The base class doesn't really have to be pure interface. Technically the constraint here is that all members have to be accessible through the object's pointer. So you can have:

pure virtual member functions (they are accessed indirectly through the virtual table)

data members (they are accessed through the object's pointer directly)

inline member functions (they are not technically accessed through the pointer, but their code is instantiated a second time in the plugin)

That leaves non-inline and static member functions. The plugin cannot access such functions from the host and the host cannot access such functions from the plugin. Unfortunately in a large application such functions can be the majority of the code.

For example, all image parsers need the CreateBitmap function. It makes sense for it to be declared in the base class and implemented on the host side. Otherwise each parser DLL will have a copy of that function.

Another limitation of this approach is that you cannot expose any global data or global functions from the host to the plugins.

So how can we improve this?

Split the host into DLL and EXE

Take a look at the USER32 module. It has two parts – user32.dll and user32.lib. The real code and data is in the DLL, and the LIB just provides placeholder functions that call into the DLL. The best part is that all this happens automatically. You link with user32.lib and automatically gain access to all functionality in user32.dll.

MFC goes a step further – it exposes whole classes that you can use directly or inherit. They do not have the limitations of the pure interface classes we discussed above.

We can do the same thing. Any base functionality you want to provide to the plugins can be put in a single DLL. Use the /IMPLIB linker option to create the corresponding LIB file. The plugins can then link with that library, and all exported functionality will be available to them. You can split the code between the DLL and the EXE any way you wish. In the extreme case shown in the sources the EXE only contains a one line WinMain function whose only job is to start the DLL.

Any global data, functions, classes, or member functions you wish to export must be marked as __declspec(dllexport) when compiling the DLL and as __declspec(dllimport) when compiling the plugins. A common trick is to use a macro:

Now the base class is not constrained of being just an interface. We are able to add more of the base functionality there. CreateBitmap will be shared between all parsers.

This time instead of the host calling a function to get the parser and add it to the list, that part is taken over by the constructor of CImageParser. When the parser object is created its constructor will automatically update the list. The host doesn't need to use GetProcAddress to see what parser is in each DLL anymore.

When g_BMPParser is created, its constructor CBMPParser() will be called. That constructor (implemented on the plugin side) will call the constructor of the base class CImageParser() (implemented on the host side). That's possible because the base constructor is marked as HOSTAPI.

Combine the host DLL and the host EXE

Usually an import library is created only when making DLLs. It is a little known trick that import library can be created even for EXEs. In Visual C++ 6 the /IMPLIB option is not available directly for EXEs as it is for DLLs. You have to add it manually to the edit box at the bottom of the Link properties. In Visual Studio 2003 it is available in the Linker\Advanced section, you just have to set its value to $(IntDir)/Host.lib.

So there you go. You have a host EXE, a number of plugin DLLs, and you can share any function, class or global data in the host with all plugins. There is no need to use GetProcAddress at all, ever, since the plugins can register themselves with the host's data structures.

The DEF file

As the host application grows bigger you will want to split it into separate static libraries. And then you are going to hit a problem.

Let's say the constructor of CImageParser is in one of the libraries and not in the main project. There is no code in the main project that refers to that function (obviously, only plugins will need to call it from their own constructors). The linker, being smart, will decide there is no use for such functions and will remove it from the EXE.

So how do you trick the linker into adding the constructor to the EXE? This is a perfect task for a DEF file. A DEF file is a text file listing all symbols that the DLL or EXE will export. The linker will be forced to include them into the output, even if no code refers to them. A DEF file can look like this:

EXPORTS
// the C++ decorated name for the CImageParser constructor
??0CImageParser@@QAE@XZ
// the C++ decorated name for CImageParser::CreateBitmap
?CreateBitmap@CImageParser@@IAEPAUHBITMAP__@@HHPAPAX@Z

To give a DEF file to the linker in VC6 you have to manually add the option /DEF:<filename> to the command line. In VS2003 you can do that in the Linker\Input section.

How do you create the DEF file? You can do it manually by listing all symbols you want to export, or you can do it automatically:

defmaker – create DEF files automatically

defmaker is a simple tool that scans LIB files, finds all symbols that are exported by the libraries, and adds them to a DEF file.

In VC6 you add this to the Pre-link step tab of the linker options. In VS2003 you do this in Build Events\Pre-Link Event in the project's options. It is going to be executed just before the linking step. Defmaker will produce the host.def file, which will then be used by the linker.

Defmaker locates the symbols by searching for the "/EXPORT:" tag in the LIB file. (Note: for some unknown reason only in VC6 service pack 6 the tag has changed to "-export:", so defmaker searches for both). The decorated C++ name of the symbol is found immediately after the tag. If the symbol refers to data instead of code it will be followed by the text ",DATA". The DEF file format requires data symbols to be marked with "<space>DATA" instead. Defmaker will convert one to the other. Probably it will be better to parse the LIB file following the official file format specs, but I have found that searching for the tags to be 100% successful.

Another use of defmaker is not related to plugins or DLLs. Sometimes you need to force the linker to include a global object even though there are no references to it. A common example is a factory system where each factory is a global object that registers itself in a list (like CImageParser does above). But if your factory object is in a static library and not in the main project the linker may decide to remove it. With defmaker you can mark the object with __declspec(dllexport) and it will be added to the EXE file.

Tip: Add the path to defmaker.exe to Visual Studio's Executable files settings. You will be able to use it from any project.

Conclusion

We've seen here how to create a plugin system without relying on GetProcAddress or trying to squeeze all functionality through interfaces. To expose any symbol from the host to the plugins just mark it with HOSTAPI. The rest is automatic. You have direct control which symbols to export and which not to.

You would write code in the plugin just as easily as writing code in a monolithic application or a static library. You can have access to base classes, global functions, and global data no matter if you write a plugin or a simple application. It is still a very good idea to have a clear separation between host functionality and plugin functionality, but it should be based on your own architecture and not dictated by technical limitations.

A word of caution - with great power comes great responsibility. You have the power to share as much or as little of the host's internals with the plugins. A key to a well designed plugin framework as with any design is finding the balance - in this case providing a simple yet powerful API. You need to export enough functionality to aid the plugin developers, yet hide the features that are likely to change in future versions or will needlessly compromise the stability of the host.

The source code

The source zip file contains four folders:

Interface - a plugin system using GetProcAddress and interfaces.

DLL+EXE - a plugin system using separate EXE and DLL for the host.

EXE - the final plugin system using a single EXE for the host.

defmaker - the sources for the defmaker tool. A compiled version of defmaker.exe is included in the root folder. It is required for compiling the EXE version of the host.

The sources contain project files for Visual C++ 6 and Visual Studio 2003. For Visual Studio 2005 you can open either project and convert it to the latest format.

Share

About the Author

Ivo started programming in 1985 on an Apple ][ clone. He graduated from Sofia University, Bulgaria with a MSCS degree. Ivo has been working as a professional programmer for over 12 years, and as a professional game programmer for over 10. He is currently employed in Pandemic Studios, a video game company in Los Angeles, California.

Comments and Discussions

im using vs2k5. I can not my app to run, i can get everything to compile and links ok. The app always crashes before im in the main() gets called. It complains and says cant find the PluginA.dll when i go to run the app. As long as i dont make any attempt to tie to another plugin with a plugin everything seems to be ok.

After reading your article twice, I still fail to see the advantage(s) over a COM-based approach. If your app is component-based then nothing prevents the plug-ins to instantiate any of the coclasses exposed by the host, the host can also easily pass a pointer to an interface where plug-ins can access the functionality they need. Your comment about the host being unable to implement functionality for the plug-in's benefit (in interface-based schemes) is unfounded.

Unlike you say in another comment, the debugger perfectly understand interfaces and COM. You can seamlessly step through a virtual method call while debugging, in-process, out of process or even step into a method running on a remote computer, and back. The editor is also perfectly COM-aware and provides intellisense for all your interfaces. The ability to run COM classes out of process if desired also provides a huge benefit in terms of stability, making it harder for a plug-in to take down the host.

Yes it is true that COM and all interface-based calls are virtual but unless you spend your time calling the empty method, this is never going to be a problem in practice. Nobody would wrap a matrix in a COM class, for example Direct3D (a large enough interface-based framework, I hope?) uses structs and statically linked methods to manipulate entities like vectors, matrices and quaternions. You can always provide the performance-critical code in a small static library (or a DLL if you really want to save space), that plug-ins can link to.

One major drawback (and showstopper as far as I'm concerned) of the approach exposed in the article is that the plug-ins are totally tied to the host, and there is little or no hope of ever being able to reuse them in different projects.

Anyway, thank you for your article, it is always interesting to read about application infrastructure / framework code.

If your app is component-based then nothing prevents the plug-ins to instantiate any of the coclasses exposed by the host, the host can also easily pass a pointer to an interface where plug-ins can access the functionality they need. Your comment about the host being unable to implement functionality for the plug-in's benefit (in interface-based schemes) is unfounded.

private:
int m_Mul;
HostInterface *m_pBase;
};
Surely you can achieve the same end result, but with twice the code. There is more complexity, more plumbing code and jumping hoops and less performance.

axelriet wrote:

Unlike you say in another comment, the debugger perfectly understand interfaces and COM. You can seamlessly step through a virtual method call while debugging, in-process, out of process or even step into a method running on a remote computer, and back.

I meant that the problem with the interfaces is that you can't see any data in the debugger (since by definition the interfaces contain no data). For example while debugging PluginClass in the example above you can't see m_Add that is in the "base" class. Also compare a COM collection with a STL vector - the vector is much easier to view in the debugger.

axelriet wrote:

Yes it is true that COM and all interface-based calls are virtual but unless you spend your time calling the empty method, this is never going to be a problem in practice. Nobody would wrap a matrix in a COM class, for example Direct3D (a large enough interface-based framework, I hope?) uses structs and statically linked methods to manipulate entities like vectors, matrices and quaternions. You can always provide the performance-critical code in a small static library (or a DLL if you really want to save space), that plug-ins can link to.

The result is that all the code in the static library is duplicated in every plugin. Depending on the specifics it can be a big deal or not. The advantage of having that common code in the host is that you can update the host (with a bug fix, or performance fix for example) and the plugins will keep working. Direct3D tries to achieve some balance by separating the functionality in 3 places - the D3D interfaces, the static libraries, and the DLLs. A new version of Direct X can fix problems with the interfaces and the DLLs, but not in the static libraries already linked with your game.

axelriet wrote:

One major drawback (and showstopper as far as I'm concerned) of the approach exposed in the article is that the plug-ins are totally tied to the host, and there is little or no hope of ever being able to reuse them in different projects.

I don't see this as a big deal. If you are writing a plugin for a specific host, what are the chances you would want to use it as a plugin for another host? Both host must be very compatible. Photoshop and ImageReady come to mind, but they are specifically designed to support the same plugins. Since you see it as a problem, can you show an example where you would want to reuse the same plugin with different projects?

In conclusion - each approach has advantages and disadvantages. Depending on the task at hand you may favor one or the other. In the next version of the article I'm going to do a more detailed comparison, based heavily on the responses posted here.

>can you show an example where you would want to reuse the same plugin with different projects

Of course. I can think of thousands of examples. Think of a spell checker plug-in. Or an image importer, an image exporter, an image rotator, an image scaler, a color management plug-in, an address validation plug-in, an emailer plug-in, a scripting plug-in... you name it. The host can be a Visual Basic script, a Windows Service, a C++ or C# app or something that does not exit yet...

Once you start thinking component-based, everything is a plug-in, and most plug-in are reusable across a wide variety of completely different applications.

Since we're looking at Visual Studio and .NET Homepage site... I don't quite understand why your solution is Win32 oriented. Possibly i don't understand something base and important because as your article title says 'Plugin System – an alternative to GetProcAddress and interfaces' - i don't see any alternative to interfaces. Well yes, in fact you bypassed GetProcAddress. But hey, COM is:1. Standartized (in fact you did use one of COM features - interfaces, though, somewhy you did use 'class' instead of MS C++ keyword 'interface')2. Complete. In my opinion COM is quite complete way to build a plugin system or any other component system.3. Documented. There are hundreds of guides and even books on how to write COM based applications.

But even this is not the point. A bit earlier friend of mine and me planned plugin architecture and we found .NET reflection a very nice solution for that.

With Reflection you can load an assembly (with plugin) into memory, find a particular class, which implements your plugin interface and run it.

Not everything on the CodeProject site is related to the .NET framework. There are quite a few articles dealing with Win32, MFC, WTL, etc. My article targets Win32 and native-code DLLs. You can't use .NET reflection with native code.

1) About COM: First, "interfaces" are not a COM feature. They are a C++ feature which COM happens to use. In the COM context and the context of this article an "interface" is a base class that has no data members and all its member functions are pure virtual. The idea is that one module can use an interface pointer to object implemented in another module without any information besides the declaration of the interface. Every interaction with the interface instance happens through virtual functions accessible through the interface pointer only. Many people are convinced that this is the "only true way" to implement a plugin system and they are forced to work around the limitations it imposes.

The article shows that this is not the case. You can provide full-featured classes that the plugins can instantiate and use directly. You can have global functions or data for the plugins to use. You can have full-featured base classes that provide many helpful services to those that inherit them. On the other end of the scale you can have pure interfaces only. And you can have everything in-between. Creating a plugin-based system vs. single app doesn't need to limit your design choices (as it is with COM). The decision which features to use is yours and should depend on the problem at hand, how you deploy your host and plugins, what degree of binary backward compatibility you want, etc.

2) Yes, COM is a complete way (but not the only way). Why bother with .NET/reflection if COM just works? A Turing machine is also complete, but you don't see that many people using it.

3) This is a moot point. If you are writing your own plugin system using COM the only part of the documentation you can use is for AddRef/Release/QueryInterface. The rest won't help much. You still need to document your API, interfaces or not.

First of all this is an excellent article describing a very useful technique.
Generally speaking, it is a bad habit to use "this" pointer in Constructor because at this point the object is not fully constructed.
In the case of ImageParser.cpp in the constructor a call to AddParser(this) happens.
When this call is being done, the actual BMPParser (or TGAParser) underlying object, hasn't been constructed, so if you call any of their members functions at that point you ll get a "Pure Virtual Function Call".
You may try it by calling
parser->ParseFile("dummy path");
in AddParser function.

Surely you can use the "this" pointer in the constructor. You do that all the time (implicitly) when you access a member. But you do that with the understanding that the object is not fully constructed and there are limitations to what you can do. Since AddParser is designed to be used in exactly this situation it should follow the same rules as the constructor. Besides, it doesn't use any members (virtual or otherwise), just the value of the "this" pointer.

The beauty is that the plugin developer doesn't need to concern himself with any of this. It is the host's responsibility to not allow something silly like this to happen - for example using objects that are not fully constructed.

That's why AddParser is entirely under the host's control. For example if the code is multi-threaded there is a danger that some other code can access the list of parsers before they are fully constructed. One solution is that AddParser registers the pointers in some private list. After LoadLibrary returns the host can move the pointers from the private list into the main list.

You are absolutely correct when you say: "It is host's responsibility not to allow something silly like this to happen".
In my case, I ve tried to make my own host(a plugin manager) which was responsible to add the plugins in a list. I have also added a "GetVersionInfo()" in the plugin interface, in order to register them with a name, check version, register capabilities etc.
And then i ve crashed onto that "old textbook" error and thought that i should mention.
Soon after, I ve realized, that this should have been a 2-step process. First get the plugins and then retrieve the details or otherwise change the plugin registration process.

Anyway your article, described a beautiful technique, saved me from a lot of work, and really thank you for that.

Host.dll is part of the host, and not a plugin. The host needs it to start up. The plugins are the .IMP files. You can remove any of them and the app will still run (but will stop supporting that image type).

The DLL+EXE version is just an example on the way to the final solution - a single host EXE that combines the functionality of the old host.dll and host.exe into one file.

Aha Thanks for the reply. I just added an x after the .imp file, and it still worked. But by removing the extension it didn't load images anymore. Nice approach, I'll try to implement your technuiqe in my StackCopy program (when finished).

i think is good enough. and it will be better if we add another 2 function to add/remove the object by ref count. just like standard COM does. if we don't sell the Dlls then is good to keep our dll small and handy.

"A word of caution - with great power comes great responsibility. You have the power to share as much or as little of the host's internals with the plugins. A key to a well designed plugin framework as with any design is finding the balance - in this case providing a simple yet powerful API. You need to export enough functionality to aid the plugin developers, yet hide the features that are likely to change in future versions or will needlessly compromise the stability of the host."

Which explains why "squeezing all functionality through interfaces" seems actually a better idea Especially if the plugin developers are not going to be the same as the host developer. And if the host developer plans to extend its functionality in the future, while still supporting plugins developed for earlier or later versions of the host.

There are pros and cons to each approach. The main purpose of the article is to show how to overcome the technical issues if you want to create plugins without interfaces. The philosophical discussion which of them is better is another topic.

That said: The interfaces impose many limitations, which are often undesirable. Imagine a big enough framework (of the size of MFC or 3DS MAX) that does everything through interfaces for the sake of it. The host can't implement base classes to help the plugins. To simulate base class inheritance with just interfaces you must create 2 objects - one on the plugin side and one on the host side. When the first object is created it must ask the host (through some other interface) to create the second "base" object. You must write a bunch of code to forward calls from your object to the base object, and to manage its lifetime. This can become very cumbersome very fast. Another problem is that interface-based code is harder to debug. The debugger can only show you the vtable pointer of the object but none of the members. Yet another problem is that every call across the plugin/host boundary becomes a virtual call. Imagine the performance hit if the matrices in MAX were implemented through interfaces.

Regarding backward compatibility: The interfaces can help in 2 ways. First, since at least on Win32 the C++ interfaces must be binary compatible with COM, you can mix and match different compilers that are COM-compliant. Second, the interfaces force you to keep the API as small as possible. Smaller API is easier to keep backward compatible. But that's almost like saying that a crappy text editor forces you write smaller (and more elegant) code.

Hmm, the reference to MAX kind of confused me. As you may know, 3D Studio MAX does provide a huge interface based plugin-framework, and an interfaced-based SDK to allow developers to re-use all kinds of functionality from its library. It's very simple: The plugin is handed a reference to the SDK upon creation - the plugin can use all of the library functions at will.

There are in fact several statements in your article that made me sad:

* The base class doesn't really have to be pure interface.
* Another limitation of this approach is that you cannot expose
any global data or global functions ...

The whole point of an plugin system is to make sure that there is a clear separation between plugin and framework. You said it yourself in the Conclusion section:

It is still a very good idea to have a clear separation between
host functionality and plugin functionality, but it should be based
on your own architecture andnot dictated by technical limitations.

How does injecting your base class code into the plugin help there. There is no separation; it's all entangled for the moment you create your class instance. It's eventually just going to create a soup of inherited code, global variables and use of public data members - the kind of code we were all doing before discovering the joys of COM based interface-programming.

As you may know, 3D Studio MAX does provide a huge interface based plugin-framework, and an interfaced-based SDK to allow developers to re-use all kinds of functionality from its library.

The plugins in MAX are not purely interface-based. It is true that much of the global functionality is accessible through a single Interface pointer implemented in 3DSMAX.EXE. However even the Interface object has its data members.

MAX also provides a ton of base classes for you to inherit in your plugins (implemented in CORE.DLL/CORE.LIB). For every method the SDK clearly states if it must be implemented by the plugin or the system. Some of the base classes even have public data - for example SimpleObject has a Mesh member in it.

MAX also implements basic data types for you to use - Matrix3, Mesh, etc. Their methods are implemented in the host and exported to the plugins with DllExport.

Disclaimer: I haven't used MAX in 4 years, so my info may not be 100% accurate or may be outdated.

Disclaimer2: I choose MAX as my initial example because it is one of the biggest plugin-based systems I have worked with. I am not citing it as the best plugin design out there, although I liked writing for it. They have chosen the pragmatic vs. the purist approach.

Bjarke Viksoe wrote:

There are in fact several statements in your article that made me sad:

* The base class doesn't really have to be pure interface.

My point here was to explain the technical and not the philosophical limitations for the base class.

There is one statement in particular in your post that made me chuckle:

The philosophical discussion which of them is better is another topic.

Agreed. I'm not going to argue about this, because I don't feel I have enough experience. I have worked with 3ds Max's plugin system since its beginning over 10 years ago and I'm sure that while it does the job, my life would have been much easier if it was done "right". So I've decided for myself to avoid (when possible) any approaches which seem "pragmatic" at first sight and give quick results, but at the price of sacrificing basic principles of software design. And while such approaches are obviously faster and better for not so large projects, I try to take the "purist" approach anyway, just for education. Again, this can be done "when possible", i.e when deadlines allow it. However, a plugin system which is expected to be used for over 10 years by thousands of people should definitely use all the time it takes to be designed properly.

Yes, I know you can do that, but that doesn't solve the problem that the DEF files solve. If your class is defined in a lib, and not in the main project, and nothing in the main project uses your class, then it won't be exported. The linker will optimize it away.