OSGi

From APIDesign

OSGi is a specification for a modular system for Java. It defines a packaging format (JAR file with additional manifest entries and associated troubles), runtime behavior, and a way to register and discover services. In contrast to the NetBeans Runtime Container, it supports package dependencies. For the rest, the systems seem similar, which has been demonstrated by the implementation of the OSGiAndNetBeans bridge.

OSGi Weirdness

OSGi and NetBeans Runtime Container are similar, however the devil lies, as always, in the details! From time to time I manage to learn more about OSGi and then I just cannot help but wonder what the motivation was behind designing such behavior! Maybe this is all caused by my preoccupation (as a designer of the NetBeans system), but I just cannot overcome the feeling that certain design decisions in OSGi are just weird.

I'll nitpick on OSGi here, hoping somebody will be able to explain the history and possibly also the rationale behind these (mis)behaviors.

Start/Stop a Bundle

I believe it makes sense to tell a module that it is going to be used or that it is used no more. Of course, I prefer Declarative Programming, but sometimes there is just no other way and some parts of the infrastructure need to be turned on programmatically. Thus, it is makes sense to have a BundleActivator or ModuleInstall (in the NetBeans case). However, then one finds that people can use an OSGi bundle without starting it!

That would be a dangerous situation. It would mean one cannot put any initialization logic into the activator/installer at all. Peter Kriens spent significant amount of his time and patience to explain me the rationale behind this behavior and possible fix:

Just add Activation-Policy: lazy into your bundle which performs important initialization in BundleActivator and it is guaranteed that as soon as somebody loads a class from your bundle, it is going to be started

This is not the default to keep things backward compatible - OK, I can understand the value of BackwardCompatibility. Through at this case it might have been better to change the semantics when the bundle manifest version was changed to two (or change it to three). The NetBeansModuleInstall is guaranteed to be called whenever somebody tries to use a module (by having a dependency on it and trying to get enabled). Although it is recommended to use declarative registrations, when activator/installer is present calling it as soon as the bundle is being used is the natural/expected behavior.

Peter (as far as I understand) rather advocates splitting the API and the implementation into separate bundles however. The service should rather be registered by independent implementation. I still don't know how my code using the API will force the system to initialize the implementation too, but I expect the answer to be OSGideclarative services (which I am not familiar with).

Possibly Eclipse is using OSGi in wrong way. The singleton initialization in activator complicates the OSGiAndNetBeans bridge. Basically it means we don't have 1:1 mapping between enabling a module in the NetBeans sense and starting a bundle in the OSGi sense. As we discover more and more OSGi based systems (like JDeveloper), we need to add more and more flexibility to cover all the possible ways people deal with the dichotomy between used and started. OSGi is very flexible and people tend to use it in many innovative ways.

At least I know now: the suitable fix is to use Activation-Policy: lazy. The only question remains: How do I convince Eclipse to modify their core bundle manifests to include this tag?

Stopping Order

Another thing related to activators is the order of starting and stopping bundles. Again, if we think about the starting to be associated with use, then it makes sense to start the modules that are used by others sooner, then starting the others. Do a topological sort based on interbundle dependencies and then start the bottom bundles first (with as less dependencies as possible). When stopping a set of bundles, do it from the top. Natural? I think so. Do you think OSGi does that? No, not at all.

I've just needed to deal with a NullPointerException caused by the fact that org.eclipse.equinox.registry is stopped sooner than org.eclipse.equinox.security when calling OSGi framework.stop(). What happened? There is a singleton in the registry and BundleActivator.stop() clears it. But still the module remains available for others to call. When the system tries to stop the security bundle, it calls into the registry trying to obtain the singleton and guess, it is not there!

Why could not the great minds behind OSGi create a specification that would sort the modules using a topological sort before shutting them down? Is that unnatural? No. But rather than that OSGi introduced start levels (which we now have to support in NetBeans). The only conclusion one can have: OSGi is not well thought out.

Range Dependencies

One of the characteristics of OSGi is use of RangeDependencies which seems like a clever good idea supporting cluelessness in its way, but if not done properly it is just a way to mess around with NP-Complete problems. And yes, there are such problems in OSGi as demonstrated by Equinox coming with a 3SAT solver.

However, beyond that, I just can't help but think that OSGi is an overdesigned, and sometimes misdesigned, technology. In addition to that, I'd like to point out that OSGi is so 2000-late. The OSGi spec 4.2 still runs on JDK 1.3... and it shows! These days Java is different. With the help of AnnotationProcessors it can do magic (and it does not need poor and misleading tricks like MagicalStrings).

I fully understand the value of pioneers. Somebody just has to be the first (well second), walk the road, and find out that this is not the right path. I believe OSGi succeeded in this mission. Everyone should understand at this point the value of modularity. Everyone should also see, however, that the OSGi path is not the right one to follow. I just hope the next attempt to standardize a module system for Java will learn from the mistakes of its predecessors (OSGi and NetBeans Runtime Container).

Let's use OSGi for interop for now, and then let's forget about it later.

Name (required):

Comment:

Defending OSGi

Neil Bartlett said ...

In the introduction you appear to be asking to be enlightened about OSGi, but then you go on to attack it from a position of ignorance and with heaps of bad attitude. Sorry but this is not the way to engage with people when you're asking for help.

If you have some specific questions about how or why OSGi works the way it does, then I and many others will be delighted to help you on the osgi-dev mailing list. Unfortunately this particular page is little more than a rant and can't be the basis for useful discourse.

Hello Neil, thanks for your comment. Yes, you are right, the page is unlikely good start for a useful discourse. On the other hand, it nicely illustrates my frustration with OSGi. Still, I'd be glad to be enlighten about the design misconceptions.

Mirko Jahn said ...

First of all, I agree that OSGi is not perfect. In fact I do not know any technology that reached some sort of complexity that doesn't have its flaws. I guess that's nature. However, I think you're missing some background/ experience in the underlying concepts and frameworks you're referring too.

First of all, start/stop of a bundle. Here it is important to distinguish between a mere library and a state aware module. For libraries it is perfectly fine to not have an initialization (static initialization is fine here as well, but no "object" related one). In the OSGi world you then do not need to create an Activator. For all modules that are in fact state aware or need some sort of initialization (like Service dependencies), you could create you Activator (or use a declarative way of expressing that). In case a module does not comply with this contract this is a bug in the bundle code. As a general rule, you only expose API's and provide the instances as services, as long as the bundle is not started, no services will be available and no harm is done. Of course there is the "ugly" overhead of needing to know what bundles need to be started and which don't (you could also start all of them - no overhead in case of library bundles, because they won't execute any code). This seems not entirely clean as an approach, but it is powerful. Just imagine you have one bundle providing the API and a simple default service with the implementation. Once you realized the implementation is not good enough, you could just go ahead, reuse the API and not the exposed service (simply do not start the bundle) and provide your own bundle implementing the service in a way you require it.

Second, stopping order. There is an order in which bundles are stopped and they are stopped in the reverse order they were actually started, so what you saw might be a timing issue or an error in the implementation. Also Eclipse is not a perfect OSGi citizen. They did a lot to improve that but they are still not there. For instance many bundles still do not use the service registry, but a more static eclipse specific construct that has flaws with the dynamism of OSGi. I guess this might explain the problem you were experiencing. Also when done correctly (with services), a stopped bundle would not expose the service any longer and the bundle consuming such service would be able to call such bean (because it wouldn't find it) - no harm could be done here. Unfortunately the developer has to account for that or us a framework that does it for him like BluePrint for instance.

Range dependencies are tricky, I agree. Also the approach on handling the correct version, updating handles in the container, refreshing the wiring between bundles. All that is not trivial and error prone. However, I haven't seen anything even close to what is possible with OSGi. You could by designing the exposed and consumed packages correctly, a powerful and future prove API. Implementers know base on version changes when they have to take a closer look (minor version change) and simple service consumers can rely on the major version as an indicator that something might have changed that they should be aware of. Especially with many modules, keeping track of changes is hell. With such ranges, it at least gives you a way to better track where to look and what to adapt in case something happens. This is still not perfect and I am not sure there is a silver bullet at all, but it's the best thing I've seen so far.

To summarize what I am trying to say. OSGi, like many other specs, is not perfect and there might be a better one in the future, but for the time being, it is pretty good and well thought through. Unfortunately the concepts are not simple and certainly not easy to adhere. Especially with the latest version 4.3, many improvements have been done to make it easier and remove potential for errors, but like every complex system, one first has to know it to fully appreciate it. ;-)

I think this is a bit off the mark. Seems like Eclipse is not really using OSGi
as designed. Proper OSGi bundles are (acc. to spec) supposed to work regardless
of what other bundles have or have not been started; that is why dynamic
services exist, and why bundles are not started in topological order. In other
words, we are working around misdesign in Eclipse, not OSGi per se. The problem
presumably does not affect Eclipse and its RCP apps because they are not
attempting to load code from an arbitrary compliant OSGi container, and so can
make assumptions about runtime behavior not guaranteed by the spec.

Reply to Mirko's Post

Hello Mirko, first of all - thank you for your time to write down such a long post here. I value your effort.

"I agree that OSGi is not perfect" - sure, I know that NetBeans Runtime Container is not perfect either. Over years in production you just have to come up with a lot of compromises which, when looking backward, may not seem perfect.

"distinguish between a mere library and a state aware module" - yes, I am aware that nobody needs to use Activator - however that is not the problem. I am puzzled by OSGi allowing to load classes from modules with Activator which has not been called! What kind of strange idea is that? In my opinion the fact that I provide an Activator should mean that whenever somebody wants to use me (e.g. activate me), I want that Activator to be called. Why this is not true in OSGi?

"ugly overhead of needing to know what bundles need to be started" - actually this may be the problem. NetBeans is worshiping so called Injectable Singletons. In this use-case, what ever API you link against means that the API is ready to work and is fully configured. The way Eclipse is using Activators violates this (which may be Eclipse fault as Mirko and Jesse indicate). You say "simply do not start the bundle" - sure, this is what works in NetBeans as well if you don't need a module with implementation of an API, just don't enable it. But still, I don't understand why there is the dichotomy between using and starting a bundle. To allow an API to be bundled with its implementation that is registered in Activator (so others can use the API without the implementation)? That sounds weird!

"reverse order they were actually started" - see, I did not know this. But then the problem is the start order. Obviously, those bundles with as less dependencies as possible, should be started first. However I understand the OSGi may have problem - there does not seem to be a bulk start operation. One can start just a single bundle, not a set of bundles. In NetBeans Runtime Container one just asks to enable a set of bundles and the system starts them in the correct order. The system always guarantee Activators are called on modules needed by somebody. Sounds like a deficiency in OSGi where determining the start order is fully left in hands of poor users of the OSGiAPI.

"certainly not easy to adhere" - if a concept is not easy to adhere, then the system is not designed for clueless use. And that is a problem as most of us are completely clueless when using others APIs. Should we threat OSGi as an assembly language for modularity hoping somebody will wrap it with higher level concepts? That would support my conclusions - OSGi helped us see the value of modularity, but proper, widely used modularity should rather not expose OSGi at all.

Sergey said ...

/ If I do anything in Linux for pmnigomrrag, I use Eclipse if I need an IDE. I also like doing as much as I can in terminal or even a note pad-like program. On Windows (for school), I really like using Notepad++, so the best alternative in Linux for NP++ that I've found is Geany.