Archive for October, 2010

As a followup of my previous post (available here) on concrete data on TCO and results of our past EU projects in the area, here is a more detailed contribution of some of the aspects that we studied in Public Administrations, starting with an explanation of my assertion that “real world TCO of an OSS migration is roughly twice the actual monetary cost” that I mentioned, and that raised quite some interest. First of all, here is a more detailed table with the tangible versus intangible costs:

On average the costs are very equally shared, with one exception in an Italian Province that had higher tangible costs (as it actually paid external contractors and services for most of the OSS activity) and a small municipality that for budget reasons shifted most of the work internally as immaterial expenses in a very significant way; the measure is also skewed by the small size of the experiment for this particular case. The measures performed in the other experiments confirm the approximate range of tangible vs. intangible; while there is some variability, the relative error is quite small.

Some interesting facts also emerged in the evaluation of negative and positive factors for adoption of OSS; in fact, this is part of the set of measurements that became the basis of our set of best practices for OSS adoption (that you can find described here, here and here). Let’s start with the negative ones:

The first observation is the fact that being in a risk averse industry sector is basically not a really significant factor (and as a counter-example, banks and financial services are substantial OSS users, and extremely risk-averse). The other factors are extremely significant, that is they explain a substantial percentage of the variability (that is, if a migration is successful or not). You can also find that most of our best practices do have a direct connection with some of the negative factors:

Perception of work under-valued if using “cheap” OSS products -> the best practice is “Provide background information on OSS: A significant obstacle of OSS adoption is the acceptance by the user, that usually has a very limited knowledge of open source and open data standards. In many cases, OSS is perceived as lower quality as it is “free”, downloadable from the Internet like many shareware packages or like amateur projects. It is important to cancel this perception, and to provide information on how OSS is developed and what is the rationale and business model that underlie it.”

Staff resistance due to fear of being de-skilled -> the best practice is “Use the migration as an occasion to improve users skill: as training for the new infrastructure is required, it may be used as a way to improve overall ICT skills; in many companies and public administrations for example little formal training is usually performed on users. This helps not only in increasing confidence, but can also used to harmonize skills among groups and in general improve performance. This may rise some resistance from the so called “local gurus”, that may perceive this overall improvement as a lessening of their social role as technical leaders. The best way to counter such resistance is to identify those users, and suggest them to access higher-level training material (that may be placed in a publicly accessible web site, for example).”

You will find that for each of these negative factors there is a specific best practice designed to help reduce its impact; taken all together, it is possible to increase the probability of success substantially with very few (and simple) methods. As for the positive factors:

It is easy to see that economic consideration are of relatively limited importance when compared with the other positive factors, and this gives value to my own theory that flexibility and vendor independence are stronger factors than economical ones (even considering that sometimes the threat of going to OSS is used to increase the discount by proprietary vendors during negotiation). Among the factors that map directly to our best practices:

Availability of OSS-literate IT personnel: “Understand the way OSS is developed: Most project are based on a cooperative development model, with a core set of developers providing most of the code (usually working for a commercial firm) and a large number of non-core contributors. This development model does provide a great code quality and a fast development cycle, but requires also a significant effort in tracking changes and updates.”

Top management support for the migration: “Be sure of management commitment to the transition: Management support and commitment have been repeatedly found to be one of the most important variable for the success of complex IT efforts, and FLOSS migrations are no exception. This commitment must be guaranteed for a time period sufficient to cover the complete migration; this means that in organizations where IT directors are frequently changed, or where management changes in fixed periods of times (for example, in public administrations where changes happens frequently) there must be a process in place to hand over management of the migration. The commitment should also extend to funding (as transitions and training will require resources, both monetary and in-house). The best way to insure continued coordination is to appoint a team with mixed experiences (management and technical) to provide continuous feedback and day-to-day management.”

Some best practices provide a reduction of the impact of negative factors and at the same time increase the impact of positive ones, like “Seek out advice or search for information on similar transitions: As the number of companies and administrations that have already performed a migration is now considerable, it is easy to find information on what to expect and how to proceed” covers both the negative “no other successful example of OSS migration in the same sector” and the positive “support from other OSS users”.

As a conclusion: there is a way to substantially increase the probability of doing a successful OSS adoption/migration, and the way goes through some simple and fact-based methods. Also, the spectre of ultra-high TCO for OSS migration can be banned through a single multiplication, to get the real numbers. If some company claims that OSS costs too much, first of all ask where they get their data from. If it is from a model, show them that reality (may) be different.

Of course, if you insist of doing everything wrong, the costs may be high. But I’m here for a reason, no?

Second important update: After quite some digging, I have finally found out that the file had the following history:

An initial import, from a private branch to the public git repo, with this commitdiff, at Sat, 10 Jan 2009 01:50:54

A deletion of the file and of most of the imported branch at Wed, 4 Mar 2009 02:28:14, with this commitdiff

The branch got re-imported, with all new files, at Wed, 4 Mar 2009 03:28:47, with this commitdiff, without PolicyNodeImpl.java

So the file was present in the git repo from Jan, 10 to March, 4; after checking the test execution code, it is clear that the code itself was not included in the final build (delivered to handsets) but was part of a test harness, that (funnily) was mostly silenced during the development period due to the majority of tests failing It is however clear that the code itself was distributed, as it was freely accessible online and through the git tree (and – funnily – it is still available under the same means). It was, however, not part of the Android SDK release, as Android 1.6 rel2 was released in December 2009, while rel3 was released in May, with the commit diff already applied.

Important update: I have verified that the code included in Android is actually a decompilation of an old Java 1.5 class file, and my own comparison is invalid, as it was done with a recent Java edition (that does have more changes – thus suggesting a reimplementation). Also, it seems that Harmony actually has not that code in its repo – despite the fact that in Android is part of the initial import under the “Harmony” subproject. My apologies.

Sigh. I had higher hopes for Oracle – while the initial complaint was dull and unsubstantial, the claims of actual copying of source code was an interesting twist (apart from the patents, of course – but that is a totally different world). Now, as ZDnet promptly writes, Oracle amended the complaint, and published the scribd link with the entire text (by the way: the HTML5 version of Scribd really, really rocks. End of parenthesis.) The relevant parts are:

40. Android includes infringing class libraries and documentation. Approximately one third of Android’s Application Programmer Interface (API) packages (available at http://developer.android.com/reference/packages.html) are derivative of Oracle America’s copyrighted Java API packages (available at http://download-llnw.oracle.com/javase/1.5.0/-docs/api/ and http://download-llnw.oracle.com/javase/1.4.2/docs/api/) and corresponding documents. The infringed elements of Oracle America’s copyrighted work include Java method and class names, definitions, organization, and parameters; the structure, organization and content of Java class libraries; and the content and organization of Java’s documentation. Examples of this copying are illustrated in Exhibit I to this complaint. In at least several instances, Android computer program code also was directly copied from copyrighted Oracle America code. For example, as may be readily seen in Exhibit J, the source code in Android’s “PolicyNodeImpl.java” class is nearly identical to “PolicyNodeImpl.java” in Oracle America’s Java, not just in name, but in the source code on a line-for-line basis.”

Hm. First of all, the definition of line-for-line equality is not correct here, as the lines are different (but quite similar). I have not developed for quite some time, but I would say that it is not strange to see similarities within the API constraints. Second, while technically part of Android, the code is Apache Harmony, a reimplementation of J2SE that tried (for many, many years) to get the compliance toolkit from Sun, but never did (and now will never do).

But the relevant point is different: the PolicyNodeImpl.java that is presented comes from the OpenJDK distribution, and was as such released under the GPL+ClassPath exception (something that is not mentioned anywhere within the complaint, by the way). Here, the claims are two and different: the first is that Android (actually, Harmony) copied its API that Oracle claims is copyrighted. The second claim is that the actual source code of the PolicyNodeImpl.java file has been copied verbatim.

Let’s start with the first one: the claim that Oracle Java APIs are protected and copyrighted. On this, it seem to me that the interface definition themselves (not the actual source code) as a mere interface does not fall within the copyright provisions, unless the actual names are trademarked, and thus its implementation requires the actual copying of a protected name in a way that is deemed incompatible by its licensee (something similar was done by Autodesk, embedding a copyrighted phrase that if not included in the file prevented the application from opening it directly). I believe that this is not the case, as the names within the source code seem to lack any “TM” or the direct use of “Java” in a way that may suggest its direct protection as a trademark.I(t should be noted that both “Java” and “OpenJDK” are trademarked, and its use is explicitly forbidden unless the code is a

And now, for the claim that the code is actually copied, here is in its glory the full diff (note: if you download the OpenJDK source you don’t get the file; I had to grab it from here in the Mercurial publication site, the raw file is this one). It seem to me that this is actually a reimplementation and not a straight copy as Oracle claims – but I would like to ask my readers for their opinion.

A disclaimer: I am not a Oracle hater, a Google or Apache fanboy, and I have no relationship with them all.

During the development of the EU Cospa project, we found that one of the most common criteria used to evaluate “average” TCO was actually not very effective in providing guidance – as the variability of the results was so large that made any form of “average” basically useless. For this reason, we performed a two-step action: the first was to define a clearly measurable set of metrics (including material and immaterial expenses) and you can find it here:
“D3.1 – Framework for evaluating return/losses of the transition to ODS/OSS”

The second aspect is related to “grouping”. We found that the optimal methodology for evaluating migration was different for different kind of transitions, like server vs. desktop, full-environment migration vs. partial, and so on; the other orthogonal aspect is whether the migration was successful or not. In fact, *when* the migration is successful, the measured (both short-term and over 5 years) TCO was substantially lower in OSS compared to pre-existing proprietary software. I highlight two cases: a group of municipalities in the North of Italy, and a modern hospital in Ireland. For the municipalities:

The slightly higher cost for the OSS part is related to the fact that an external consultancy was paid to provide the support. An alternative strategy could have been to retrain the staff for Linux support, using consultancies only in year 1 and 2- leading to an estimated total cost for the OSS solution exactly in line with the proprietary one. The municipalities also performed an in-depth analysis of efficiency; that is, documents processed per day, comparing openoffice and MS office. This was possible thanks to a small applet installed (with users and unions consent) on the PC, recording the user actions and the applications and files used during the migration evaluation. It was found that users were actually *more* productive with OOo in a substantial way. This is probably not related to a relative technical advantage of OOo vs. MS office, but on the fact that some training was provided on OpenOffice.org before beginning the migration – something that was not done before for internal personnel. So many users actually never had any formal training on any office application, and the limited (4 hours) training performed before the migration actually substantially improved their overall productivity.

On the other hand, it is clear that OOo is – from the point of view of the user – not lowering the productivity of employees, and can perform the necessary tasks without impacting the municipality operations.

- Hospital:
The migration was done in two steps; a first one (groupware, content management, openoffice) and a second one (ERP, medical image management).
In the first, the Initial acquisition cost was: proprietary 735K€, OSS 68K€

The hospital does have a much larger saving percentage when compared with other comparable cases because they were quite more mature in terms of OSS adoption; thus, most of the external, paid consulting was not necessary for their larger migration.

Some of the interesting aspects that we observed:

In both tangible and intangible costs, the reality is that one of the most important expense is software search and selection, and the costs incurred in selecting the “wrong” one. This is one of the reasons why in our guidelines we have included the use of established, pragmatic software selection methodologies like FLOSSMETRICS or QUALIPSO (actually we found no basic difference in “effectiveness” among methods: just use at least one!)
This information is also something that can be reused and disseminated among similar groups; for example, the information on suitability of a backup solution for municipalities can be spread as a “best practice” among similar users, thus reducing the cost of searching and evaluating it. If such a widespread practice can be performed, we estimate that OSS adoption/migration costs can be reduced of something between 17% and 22% with just information spreading alone.

On average, the cost of migration (tangible vs. intangible) was nearly equal with one exception that was 27% tangible vs. 73% intangible, due to the pressure to use older pcs, and reuse resources when possible for budgetary reasons. In general, if you want to know the “real” TCO, simply take your material costs and multiply by two. Rough, but surprisingly accurate.

Both in COSPA, OpenTTT and our own consulting activity we found that 70% of users *do not need* external support services after the initial migration is performed. For example, while most of COSPA users paid for server support fees for RedHat Enterprise, a substantial percentage could have used a clone like Centos or Oracle linux with the same level of service and support. Also, while not universally possible, community-based support has been found sufficient and capable in a large number of environments. A problem with community support has been found in terms of “attitude”; some users accessed the forums with the same expectations of a paid offering, seriously damaging the image and possibility of support (something like “I need an answer NOW or I’ll sue you!” sent to a public support forum for an open source product). For this reason, we have suggested in our best practices to have a single, central point of contact between the internal users and the external OSS communities that is trained and expert in how OSS works to forward requests and seek solutions. This can reduce, after initial migration and a 1-2 year period of “adaptation”, support costs by shifting some of the support calls to communities. This can reduce costs of a further 15-20% on average.

I am quite proud of the work that we did on EveryDesk – a full desktop as a bootable USB key, fully modifiable and adaptable. We are using it in schools, public administrations and companies, where the increased efficiency of Linux makes a difference in making old computers usable again – or helping in the problem of managing PCs that are remote or in hostile environments.

However, this is not enough. You may be without a USB-bootable computer, or you may be using a tablet like an iPad or a Galaxy Pad (something that I see more and more everywhere). In these environments, you may need something more powerful than the apps that are available there – a full Office-like application, or a real desktop browser to access a corporate banking application; maybe you need a specific client for older systems, like the IBM iSeries (the old AS/400) or some special client in Java – on system that do not have java or flash.

For this kind of applications, we are working on a system that embeds a full HTML5 desktop in a FaceBook application, making it accessible from any recent web browser, including the iPad. This way, you can have a full desktop everywhere you go. We hope that it can be of interest; as soon as it is ready, we will release source code and blueprints.

We have prepared a small demo of how it works right now; it is a real screen capture from my own personal EveryDesk/Online instance, done on a normal ADSL line. It should give an idea on how it may work for you.

There is something that I mentioned many, many times: EU projects tend to talk about Open Source, but it is sometimes difficult for project managers to really grasp what OSS is, and how it can be used for real – not only during the project lifetime, but afterwards as well. For this reason, with my thanks to the EU project officers in the Internet of Services group for the invitation, I have prepared a small guide on how to engage open source project, how to evaluate the best exploitation strategy, how to select a business model, and (more important) a simple and pragmatic approach on selecting an OSS license for a new project. The guide will be presented at the Internet of Services 2010 event; the collaboration meeting is in Brussels, tomorrow, and the open source part will start from 11:30. A detailed agenda is available here; for more information, the event webpage is here. Just after the end of the event, the draft of the guide for FP7 project will be mirrored here as well.