why I’m not using a great eclipse project …

remark: the goal of this blog post is to report my experiences from my very personal POV and to help that projects can be more attractive to developers and perhaps also helping some developers how to solve problems if the TargetPlatform doesn’t work as expected

if you’re a member of an Open Source project perhaps you know this phenomen: someone is new on your newsgroup, asks questions and suddenly he/she disappears. there’s a chance that he/she isn’t a satisfied consumer of your project.

let me tell you a short story from my experiences last days:

as you probably know, I’m working on enterprise and mobile solutions, where for the enterprise site I prefer Eclipse RT projects like Equinox, Riena, … and at mobile site at the moment I’m focused on BlackBerry Java development. (not forget to mention Eclipse Modeling projects like MWE, Xpand/Xtend or Xtext helping me to solve my requirements to develop great apps in a short timeframe)

if developing mobile Business APPs, then you not only have a mobile APP – there’s always the need to communicate with a server and most times you have to support a bunch of protocols. The task I had to solve was communicating to a server from a mobile using TCP Socket. on BlackBerry devices this can be a ‘direct’ TCP Socket connection from the device to a server or a connection thru BES (BlackBerry Enterprise Server)

what do I need for Socket Connections:

should fit into my Equinox OSGI server

easy to be added to my TargetPlatform

using Declarative Services to be flexible

EPL licensed because I always provide core parts of my work under EPL

I googled to see if there’s a ready-to-go OSGI solution and found some TCP Socket servers but not OSGI – ready and in many cases not compatible with EPL.

but hey – I know that TCP Socket isn’t the only connection I need – there’s this great Eclipse Project supporting all kinds of communication: ECF (Eclipse Communication Framework)

at first I visited the ECF project homepage to see if there’s such a thing like a simple TCP Socket Server available but didn’t found one, then I asked in the NewsGroup and got an answer very soon: yes, ECF is using TCP Socket in many cases and I should take a look at the Generic Server.

because working on RT projects I need all to be available in my Target Platform. this should be easy for an eclipse RT project (I thought)

looked at the project sites to find installation docs how to do this but only found the update site.

to avoid all side effects I started with a fresh 3.6.1 EPP Modeling installation and created a new TargetPlatform based on the IDE, then I added the update site to my Target Platform definition. at first I checked that required software should be included.

unfortunately this runs into errors:

ok – nice try but then let’s try it without checking “include required software” – but this doesn’t help because after setting the Target Platform the ECF plug-ins are not resolved.

because there’s no documentation and I’m not getting answers to this from the newsgroup I started to try out some combinations.

btw: did all work using the normal update site and also the nightly builds.

1. log4j: I don’t want to discuss now if required bundle or import package is better, but there are some good reasons for dependencies to logging frameworks to use import package because you never know which bundle will provide the log4j packages. in my case I’m using SLF4J where’s a bundle available to do this. ok – I opened a bugzilla (335784) to change this and there’s some work on this. for now I solved this and imported the bundle org.apache.zookeeper into the workspace, opened the MANIFEST.MF and changed it. also I added my SLF4J and Logback bundles.

2. the missing javax.xml: thats also easy to solve. perhaps you know that you can add bundles on-the-fly to your Target Platform without knowing who’s providing it. You can ‘Add Artifacts’ (on the Mac CMD-ALT-SHIFT-A to get this view:)

3. missing constraint: org.eclipse.ecf.osgi.services.distribution tries to import the package org.eclipse.ecf.core.util in version 3.2.0. This package was provided by org.eclipse.ecf bundle, but this bundle exports the package without a version number so it couldn’t be resolved. again as a good citizen I opened a bugzilla (335786). to help me in the meantime I imported org.eclipse.ecf.osgi.services.distribution into the workspace and removed the version from import-package-dependency. this was fixed very soon from ECF – so if using the N-Builds it should be there soon.

great: all ECF PlugIns resolved – so I created my OSGI launch config and really I got the Generic Server running 🙂

now the real work can start: to see how to get a TCP Socket Server running from ECF. …worked thru the code of GenericServer and found that there’s a ServerManager created in Activator. worked thru this code to find out where I can add my classes to get the Socket Connection and work with ServerSocket and Socket. found out that I can define properties (like port number) in xml or extension registry. decided to use the xml – but trying to connect from outside doesn’t work. found out that there was a bug in an if-else construct and the xml was never read. again opened a bugzilla, added the solution and got soon answers inside the bugzilla.

but I wasn’t satisfied to hear ‘this ServerManager is outdated, only for backword compatibility, we know there’s no documentation but we have no resources…‘

uuups: no time to declare it as deprecated ?

THIS was the point I decided to stop evaluating ECF and started to write an (OSGI aware) TCP Socket Server by myself. will blog about this and provide sources.

I spent more time trying to find out what happens inside ECF (without finding the solution) then developing it by myself.

ECF is a great project and I can imagine that you can do all what you want, but from my POV it’s more important to have documentation to guide new users then always adding new features or changing the way how it works.

comparing my experiences with another RT Project like Riena – there are lightyears between Riena and ECF.

Riena provides

well documentation

installation-hints-to-get-target-platform running

many different examples to be run as eclipse project or client-server

wizards to create your own project based on Riena technologie

uncount unit tests

…and Riena is also a very complex project …but it’s really fun

remark: the only goal of this blog post is to give the ECF team some hints to become better. it also may be that I don’t found the right informations from wiki. and maybe if you’re looking for another kind of communication it will run out of the box from ECF.

the good things: work on my reported bugzilla happens soon after reporting – even on the weekend – thx to the ECF team
—————————————————————————————————————————

Have you considered the Net4j Signalling Platform? From my understanding (correct me if I’m wrong) ECF focusses more on integration aspects than on good TCP transport. Net4j is NIO-based, has TCP, HTTP and JVM transports built-in (SSL being contributed right now), does not depend on any third party components, supports multiplexing different application protocols through a single connection (e.g. socket), and, and, and… 😉

I completely don’t get why people ship stuff that takes a good deal of black magic to even install. I especially don’t get why people choose to release stuff that’s developed against non-stable/milestone releases of dependencies, because then you’re almost certain that it will break in about 2 months time. Again, this is a great argument in favor of continuous deployment because you’re ironing out the problems related to those kinds of dependencies, both across time and across machines.

For myself, I have the habit of Javadoc-ing everything as soon as possible, if only for my own sake (“What was this class doing again and in what respect is it different from that other class?!”). If complemented with documented typical usage patterns or unit tests that exercise that class in a representative manner. Together with a small overview of the entire project and where to start looking, this should be enough to get most people going.

red-open isn’t out there yet because we have to finish documentation, installation, videos etc.
redView is 1.0 since end of december 2010, we have example projects for each feature, installation guides, videos, javadoc, more then 1000 unit tests and are just working on an online – workshop based on a one-day-tutorial at oop conference.
if you miss something please report this, because we know that we’re not perfect.
ekke

This posting exemplifies a primary problem with some in the Eclipse community: hacks like you complain ignorantly about things not being available (like documentation for not-officially-deprecated-but obsoleted ECF generic server usage) and then contribute *NOTHING* to the solution.

Also your dismissive response to my comments about documentation and project resources (which is deceptively misquoted in the article, btw…here’s the actual quote: https://bugs.eclipse.org/bugs/show_bug.cgi?id=335848#c6 ) pretty clearly demonstrate that you are engaged in doing a hack job on ECF with this posting. I can’t imagine why that would be (sic).

hi scott,
first time in life that someone said to me that I’m ignorant.
I couldn’t know that the ServerManager was obsolete because I was told to take a look at the GenericServer.
and I reported bugzillas immediately, discussed, helped, send solution… – think this is what a contributor (new to ECF) does 😉
I’m working so many years with eclipse projects and always try to help and am spending much time for bugzillas and testing things out.
…I still think that my critic was in positive sense – if I’ve choosen wrong words – excuse me – english isn’t my native language
ekke

hi scott,
let’s drink a beer at EclipseCon …
if you read my blog again – it’s not negative:
now other users trying to use ECF in TargetPlatform know what they have to do to make it run
and you know why I’m not using ECF at the moment – will try it out again some months later
it’s always good to know why people aren’t satisfied
ekke

Your blog is indeed negative…it paints ECF in a very negative light…e.g. the target platform issues…even though many of the problems you experienced with tp resolution are not exclusively ECF’s fault…as well as the lack of documentation and apparent lack of examples (which you just didn’t find).

Of course you didn’t mention that there is in fact quite a lot of examples and getting started documentation for remote services/server usage (which is likely at least related to your use case). But of course you didn’t go to the trouble to ask about any of that.

When you start contributing rather than criticizing and publicly misrepresenting…as this blog posting has done…*then* I’ll go back to considering your comments worthwhile. Until then, I’ll focus on the addressing the needs of the ECF community (since unlike many EF projects we actually have a community of consumers)…to the limit of my resources.

Here’s a good example of how the misinformation in this blog posting is damaging to ECF.

First, ECF and Restlet are (currently) not even in the same ballpark…in fact, there is an effort (which we are struggling to resource, of course) to *use/integrate* Restlet, the ECF Rest api, and ECF remote services together…to allow people to easily create Restlet-based OSGi remote services.

Oops – Apologies if my miss-worded post gave you and others the wrong idea. Ekke’s post came AFTER my decision (made this morning) NOT before it. But there’s more to it than that…

As an old network analyst from way back I’ve been looking at the ECF project for sometime now and I think Wow, that’s just what we need for Eclipse. While I have done some work with servers and clients and was even a minor comitter on the Apache Mina project for a short time where I tried to introduce OSGi to those good folks, I have not had the requirement (yet) to utilize a full communication framework like ECF. Heck its a lot more than a communication framework it’s a full suite of integration tools for communicating with remote services and non OSGi services. I fully respect what you are doing and know your efforts are global in scope. (I especially like all the bundled discovery tools – Jini being one of my previous interests.)

So I have done some one offs on the networking side, and I now have requirement for something a little bit more than me doing rifts on the org.osgi.service.io.ConnectorService. I need some rest;-)

So of course Restlet is not in the same ballpark ECF. But it seems it would like to join the team.

So my decision was a temporary driven by time frame one. It is my hope that when my requirements expand to the tough stuff like discovery, I will be able to move everything I have done in the short term to ECF.

believe me – my post was no post “against” ECF. I know its a great project providing so many communication protocols. I only reported that in the small part I tested I ran into so many problems because of missing docs and getting informations leading me to the wrong path…
thats the only reason – then I described how to setup a target platform to make it run (to help others perhaps missing this information)
I tested 2 days and reported bugzillas to help.
…and finally got the information that I looked at ‘outdated’ classes – but this wasn’t mentioned anywhere. so I spent time for nothing and this is frustrating.
I hoped that my report will help you.

if you follow my blogs then you should know that I not only talk about the good things happening in daily work.
you can ask Eike per ex. 😉

I’m not sure why, but there doesn’t seem to be a ‘reply’ button on John’s posting, so I’m going to manually include it here, so I can respond to specific sections.

[John]
Hi Scott,

Oops – Apologies if my miss-worded post gave you and others the wrong idea. Ekke’s post came AFTER my decision (made this morning) NOT before it. But there’s more to it than that…

As an old network analyst from way back I’ve been looking at the ECF project for sometime now and I think Wow, that’s just what we need for Eclipse. While I have done some work with servers and clients and was even a minor comitter on the Apache Mina project for a short time where I tried to introduce OSGi to those good folks, I have not had the requirement (yet) to utilize a full communication framework like ECF. Heck its a lot more than a communication framework it’s a full suite of integration tools for communicating with remote services and non OSGi services. I fully respect what you are doing and know your efforts are global in scope. (I especially like all the bundled discovery tools – Jini being one of my previous interests.)

So I have done some one offs on the networking side, and I now have requirement for something a little bit more than me doing rifts on the org.osgi.service.io.ConnectorService. I need some rest;-)

So of course Restlet is not in the same ballpark ECF. But it seems it would like to join the team.

[Scott]
Yes. It certainly does seem like it would like to join the team. There have been others that have requested Restlet integration with OSGI remote services (via ECF’s impl of OSGi remote services…think how nice that would be…create restlets and have the automatically published as OSGi remote services…for discovery, using Remote Service Admin standards, etc).

But here’s the rub. Lacking anyone else to contribute work to this new capability, how are we (ECF) to get it done? I could do it in ~ two weeks worth of my focused/full time…but I can’t sponsor even my own time to do this currently…and ECF isn’t a corporate controlled project like most of the other projects at EF. At the same time, people like Ekke complain about us not having enough documentation for the stuff that we already have (which is admittedly true…we don’t have enough documentation…although I still think that Ekke’s blog posting is designed to make the project look worse that it is in this respect).

There’s no free lunch…but everyone seems to assume that ECF is an inexhaustible fountain of new features, documentation, support, etc., etc. I can tell you (as one of the 3 actually active committers)…it’s not.

[John]
So my decision was a temporary driven by time frame one. It is my hope that when my requirements expand to the tough stuff like discovery, I will be able to move everything I have done in the short term to ECF.

[Scott]
What would be great, I think, would be if you/others could get what you want…e.g. to use Reslet RPC along with OSGI/ECF remote services…with transport-independent discovery (which ECF has), with transport-independent distribution/rpc (which ECF has), along with OSGi standard implementations of the publishing and accessing APIs (which ECF has with remote services and the new remote service admin).

But since you as well as other consumers have a variety of needs, this is going to take some community commitment…either to support such integrations (i.e. with Restlet)…e.g. by sponsoring committers to implement specific things (Restlet integration, documentation, examples, OSGi TCK testing, Virgo integration, more discovery providers, etc) or to contribute them directly to the project. It can’t ride exclusively and permanently on the backs of the ECF committers (as it does now).

[John]
again I am sorry if my fat fingering gave the wrong impression.

[Scott]
I understand…no apology from you needed. My main irritation is that Ekke’s posting paints ECF in an unfairly negative light and is damaging…as it puts consumers of a project off.

Again, Ekke’s posting had no ‘reply’ link, so I’m reproducing and responding to it here

[Ekke]
Scott,

believe me – my post was no post “against” ECF. I know its a great project providing so many communication protocols. I only reported that in the small part I tested I ran into so many problems because of missing docs and getting informations leading me to the wrong path…

[Scott]
Right…that’s my problem with your posting…that you only reported the ‘small part that I tested’…without even asking anywhere (on bug, on ecf-dev, on newsgroup afaik) about the availability of other documentation, examples, target platform (e.g. and with the target platform install…why didn’t you just uncheck include required?).

[Ekke]
thats the only reason – then I described how to setup a target platform to make it run (to help others perhaps missing this information)
I tested 2 days and reported bugzillas to help.

[Scott]
and then you posted in your blog about the problems/bugs and didn’t mention that these were now *fixed*…in direct response to your bug reports. You made it sound as if we were idiots for not finding these problems previously…and then hadn’t fixed them.

[Ekke]
…and finally got the information that I looked at ‘outdated’ classes – but this wasn’t mentioned anywhere.

[Scott]
Then maybe you should create a wiki page that mentions it. Surely that would be more helpful than publicly posting

…or why documentation is important 😉

[Ekke]
so I spent time for nothing and this is frustrating.

[Scott] Frustrating? Try spending more time than you can imagine creating docs, creating examples, doing most of the coding, providing support for people’s use cases/bug reports only to have someone that didn’t even ask about the availability of these things criticize the project simply because they didn’t get exactly what they wanted with absolutely no effort on their part. That’s frustrating.

[Ekke]
I hoped that my report will help you.

[Scott]
I just don’t believe that this is what you intended. If you really wanted to help, then why didn’t you just *help* (e.g. by contributing documentation, examples, startup docs for the new IGenericServiceContainerFactory service, etc). Next time, my suggestion would be: if you want to help, then help. If you don’t want to help, then complain about stuff in an effort to denigrate the project.

[Ekke]
if you follow my blogs then you should know that I not only talk about the good things happening in daily work.

[Scott]
Previously I read your blog occasionally…but I don’t think so anymore. My main comment is that you should spend less time complaining about things that don’t work as you expect and more time contributing to make them work as you expect…it might actually result in you getting more software and support for free to build your product. Or you can build everything yourself.

Since our project resources are so low, most of our support and question and answer resources are necessarily via the ecf-dev mailing list (we can’t monitor the newsgroup very often). I try to make that clear on the newsgroup as often as possible.

>we have 2 different POV’s and we should continue this face-to-face – thats better

Hi Scott,
>we can’t monitor the newsgroup very often
I must have overlooked that communication should go thru dev-list –
if evaluating projects I’m always using the Eclipse Community Forums / Newsgroup – the devlist is the 2nd step after decision done
so next time evaluating I’ll use the devlist
ekke

re: Since our project resources are so low, most of our support and question and answer resources are necessarily via the ecf-dev mailing list (we can’t monitor the newsgroup very often). I try to make that clear on the newsgroup as often as possible.

I also had the poor pleasure to talk to forums that nobody listens to. Not only your newsgroup! Not only Eclipse forums! From a user perspective this is just not acceptable. If no project responsible moderates a forum, please close it.

You may argue that users can help themselves there, but why shouldn’t they be able to do that in your mailing list?

One might argue that would flood your dev-list. Granted. But that takes us back to why we do have two separate forums, newsgroups for user support and dev-lists for developer discussions.

This way or that way, there’s just no good reason for having a user help forum where users can not get help when they need it.

FWIW. ECF 3.5 will be coming out week of March 14, 2011. The install issues with ECF 3.4 you identified in your original posting were dealt with to the degree possible. Explanation: Because of a bug in PDE, it’s not possible to have ‘Include required software’ checked…the default in the PDE target editor…and install ECF target components without p2 dependency errors. This is because of the presence of ECF core in the Eclipse SDK (filetransfer for p2), the dependence of the rest of ECF on ECF core, and feature versioning.

WRT documentation, we’ve created a wiki page to document how to install ECF/Remote Services in Eclipse or the Target Platform. Given resources, this is the maximum that we can currently do to reduce the complexity in using the PDE target platform editor to add either all of ECF or just ECF Remote Services to a target platform.

Also, WRT ECF generic server usage, additional documentation has been added here, and as javadocs on the referenced api in org.eclipse.ecf.server.generic.