Most of the feedback users send to us privately is good, but not all of it. We do get some negative feedback every now and then, though we try not to get too worked up about it. In a way negative feedback is good too, because at least a user cared enough to bother to shed light on an issue that was troubling them. We can (and do) resolve most issues users commonly report to us by making technical fixes to TurnKey, but sometimes users complain about things we can't change. Except perhaps to try and explain our thinking better.

For example, shortly after we came out with a TKLBAM related announcement, one user sent us a private e-mail with the following complaint:

TKLBAM is not anything I would be interested in ever. I do not want to jump to conclusions as I love the work you are doing but are you somehow sponsored by Amazon? I have no desire to use another company's cloud.

I hope you move onto other things soon and move away from what at least to me seems like the end of your open source days and moving to making money.

I responded as follows:

Thanks for your feedback. We're not currently sponsored by Amazon, though it would be nice if we were!

We started with support for Amazon Web Services because they're the leading public cloud vendor and have the richest API. In the future we'd like to support other public clouds and eventually private clouds as well.

Also, you don't have to use Amazon if you don't want to. If you read the FAQ carefully you'll notice that TKLBAM works with any storage target you want (e.g., filesystem, NFS, rsync, SSH, etc.). It's just a bit more difficult to use without the cloud infrastructure.

I can understand how some people aren't too enthusiastic about the whole cloud concept. I personally don't think we should view the cloud as the end-all solution for all computing woes, but rather as another tool in our belt. A tool with pros and cons. Like any other.

What really puzzled me was that it sounded like our user was inherently hostile to the notion that an open source project could have any sources of revenue to sustain it at all, as if business and open source were two opposing sides in a zero-sum equation.

But most open source projects are funded by providing various commercial services (e.g., support, training, customization) to large companies. That's where the money is. Most large companies wouldn't touch a solution that wasn't commercially backed somehow. They just don't get how that could work.

So I have a hard time imagining any individual, regardless of political bent, being ideologically opposed to the funding of open source software by for-profit entities. It makes about as much sense to me as being opposed to business funding of common-good causes such as environmental protection.

In my opinion, this sort of "us vs them" thinking is so out of touch with reality it's not even wrong.

To the best of my knowledge there's nothing inherent about open source that makes it incompatible with business. Companies such as RedHat, Canonical are good examples of how you can make it work. There are many others.

We've come a long way since Eric Raymond explained how open source works to the business world in the Cathedral and the Bazaar. Today most open source code is written by developers who write open source software for a living. That implies employers who view open source development as financially sustainable. It's a win-win situation, not a zero-sum game.

The end result is more people working on open source full-time, more open source software, and a richer ecosystem. It's not a pie that needs to be divided, but a tide of technological wealth that lifts all boats. How can this be a bad thing? Even Richard Stallman the Free Software Foundation, free software purists, encourage people to earn a living from open source software:

Many people believe that the spirit of the GNU Project is that you should not charge money for distributing copies of software, or that you should charge as little as possible — just enough to cover the cost. This is a misunderstanding.

Actually, we encourage people who redistribute free software to charge as much as they wish or can. If this seems surprising to you, please read on.

The word “free” has two legitimate general meanings; it can refer either to freedom or to price. When we speak of “free software”, we're talking about freedom, not price. (Think of “free speech”, not “free beer”.) Specifically, it means that a user is free to run the program, change the program, and redistribute the program with or without changes.

My personal take on this is that it can be highly beneficial, especially for large open source projects to have some kind of business model. Otherwise there is a strict limit to how much resources you can put in. Having a way to pay the bills is a good way to ensure long term sustainability.

The tricky part is finding just the right balance between commercial interests and open source ideals. Many projects that give commercial open source software a bad name fail to find the right balance and lose the community's trust as a result. In the long term this can be fatal because engaging the community is essential for a successful open source project. Otherwise you get all of the disadvantages of proprietary software (e.g., no outside contributors) with none of the advantages (e.g., various "IP" protections).

Nobody minds TKL making money. It's the change in direction that bothers people. TKL's main focus was on producing useful VM apliances. When the cloud services work started, the appliances went by the wayside.

Typically, open source makes its money through support. Contracting out to companies that want the software supported on their systems.

Thanks for the feedback. Harsh but honest. I can understand where the
shifted-focus perception comes from. The cloud stuff is the only thing
that has really been in the forefront the last year or so.

There's a few good reasons for that:

Last year TurnKey lost its funding from my ex-employer and one of our
sponsors. That wasn't a surprise. We could see it coming.
Unfortunately, without funding it soon became evident that the
project would not be sustainable over the long haul without some way
of generating at least enough revenue to fund the regular maintenance
work that needs to happen for TurnKey to support its infrastructure
and issue regular, timely releases.

This is another way of saying we realized we hit a bottleneck. If we
prioritized expanding the appliance roster we could find ourselves
working full time just to stay in the same place.

We made a painful strategic decision and decided two things had to happen simultaneously. The project
needed to open up so that it didn't depend so much on the core team
and we would need to find a viable source of long term funding. One
that didn't conflict with our open source charter (I.e., the
appliances would stay 100% free). We didn't want the rug pulled from
under us again.

Support wasn't really an option as the economics don't work out if
you don't have sufficient scale for it. Also, TurnKey currently
caters more to the low-end of individuals and small business rather
the big businesses that are the traditional customers for support.

We did start talking to Canonical about providing support in the future for
TurnKey but cooperations with big companies like Canonical move at their own sweet time. Even if that developed more quickly we didn't want the project's future to be at the mercy of an untested relationship with a single third party.

At least in the short term, a revenue stream related to cloud
services was the only practical option.

Spinning off TurnKey as an independent legal entity was a big
distraction. We don't have much experience with that sort of thing.
Nobody held our hand and told us - this is what you need to do (or
better yet, let me do it for you). We had to navigate the maze
ourselves. Believe me, there is absolutely no pleasure in having to
pay attention to all this business / legal / financial stuff when we
could be writing code. It's kind of a nightmare actually. Did I
mention how I hate lawyers? Thankfully, that's finally behind us now.

We've prioritized rehauling our infrastructure from the ground up so
we could open up the project to everyone. We realize it's untenable
for an open source project to depend so intimately on a closed team
of developers. TKLPatch let's you customize existing appliances but
it isn't powerful enough to rebuild an appliance from the ground up.
What we've been using behind the scenes is powerful, but it's
unnecessarily complex, unreproducible, and ultimately unfit for
public consumption. We're going to change that.

We want the build process to be available to everyone. We want
appliances developed under publically available revision control. We
want people to be able to fork Core (or any other appliance), build
their own version, choose the build type, get the infrastructure to
build it for them, launch it in the cloud if they want, etc.

It's a big project for us and most of it is back-end work that hasn't
yet seen the light of day. But TurnKey's future depends on us making
it happen even it means delaying releases.

Anyhow, some of this is a moot point as the viability of the cloud
services depends on the appliance library. We don't harbor any illusions
as to what most people come to the project for. It's not cloud services.
That just happens to be the only way the project can pay for itself.

Now that the cloud service infrastructure is in place TurnKey has the
beginnings of a source of sustainable long term funding. But for it to
grow we're going to have to reshift focus to the main thing people come
to TurnKey for: the appliance library.

Be patient. We're doing our best with very little in the way of
resources.

A lot of this post is news to me! So you've lost your primary backer?!? That puts a whole new spin on the current TKL-state-of-play! (For me at least!!) If you've been dealing with all that legal hoo-har then no wonder new appliance release has been sidelined!

It sounds almost a bit precarious because as you say people come to TKL for the appliances, but without the cloud stuff the future of appliances is in doubt! But because of the time/energy consumption of building the cloud stuff, appliance production has suffered. Same for releasing new appliances and allowing greater community involvement in the creation process. Without it; it places too much burden on you guys. But to get there requires a huge burden now (and appliance production suffers short term...!

I'm sure that behind the scenes you have a lot of work done and I suspect that you already have some sort of alpha prototype? It actually occurred to me last night that a Hub type/based process WebUI (backed by AWS and/or cloudtask no doubt) that provided Core images and a range of official and community provided/approved patches (and probably the facility to upload your own custom ones too) that could then spit out an image of the desired type and upload it to SourceForge (assuming the same image doesn't already exist). TBH that's pretty much what I'm thinking about doing to streamline upload and update of the 'TKL Community Download' SF stuff. Although I probably won't bother with a WebUI (it's just easier for me to write a quick and dirty bash file - it won't really be for general consumption - although I'll probably host it on GitHub now I'm geting the hang of that).

Anyway I'll be back later to provide a more on topic response to the blog post. Hope you're keeping well Liraz :)

There were a lot of things that weren't really clear even to us up until
a couple of months ago. Just dealing with that stuff was painful enough
without having to try to explain exactly what was going on behind the
scenes to the community. So rather than promise that everything would be
OK, we just worked hard to make sure it really would be.

Just to make this clear: there was never any question regarding whether
TurnKey would continue to exist. We've put too much of ourselves into it
to let go. We're here for the long haul. In the worst case scenario we
would have kept TurnKey on life support from our own personal funds
while we figured things out. But we want TurnKey to do much more than
survive.

Anyhow we have a ton of work to do but the state of emergency has
passed. Maybe that's why I feel more comfortable talking about it
openly. TurnKey is already self-funded, at least to the degree that it
can pay for all of the infrastructure expenses. We have reason to
believe the funding from the service fee for EBS-backed instances will
be enough to start expanding the project's development resources a few
months down the road, after we finally make good on our promise to
release more appliances, 64-bit support, etc.

Regarding the cloud infrastructure, we are close to something we can
start using ourselves for the next batch of appliances. Once we hit a
prototype that works reasonably well we will start revealing it to the
community. The web UI is important in long run but its actually not our
top priority (yet). For developers, getting the back-end functionality
to work right is more important and we can provide a command line
interface for that (e.g., like CloudTask).

I just talked to Alon and he said from my comment it wasn't clear why
TurnKey lost its funding last year and that maybe I should clarify that
for those that don't know the story.

It's really very simple. About a year and a half ago it was finally
decided by my ex-employer that TurnKey was taking up too many resources
to be be written off as just a fun side-project of no real consequence.
If we wanted to continue to put (and even increase) resources into it,
TurnKey had to make its own way and that would have to be done within
the frame a new legal entity. Sort of like the organizational equivalent
of your parents forcing you to become independent by kicking you out of
the basement. :)

Regarding the former sponsor - it was a backup company, and they cut
funding the very day we released TKLBAM. I can totally understand how
they wouldn't want to fund an open source project that made their backup
software redundant.

We weren't too broken up about it as we say both coming, but even though
TurnKey had modest expenses, the donations didn't cover them and it was
easy to see where this was going. We had to put out the fire before it
burned us down.

TKL is a real ecosystem. It is also, potentially, a complete alternative to the other ecosystems that have dominated so much of computing the last 25 years or so (think Microsoft, Novell, Apple and others).

There are other "appliance focused" groups out there, Bitnami comes to mind, that charge a subscription for access. TKL's having appliances out in the open and avaialable has been more than commendable. Yes, the TKLBAM program is pretty hardwired to Amazon, but it is very reasonably priced (at least it has been) and purportedly it is pretty easy to backup to just about anywhere (lets work on understanding this more...)

Offering a cloud service, like the backup service, where the organization (however it is defined) gets some income seems like a reasonable endeavor. Hosting systems in a CoLo, in your house or just about anywhere else costs money too. And the mechanism and ability (I am not sure about the value yet) on the TKL Hub is really really easy to work with.

You've got to make some money if you are spending the majority of your time on this. My recomendation only is to keep the price as low as you can but make it at the same time really, really scalable (I am sure it is). This can be a very big part of the landscape.. and it can and should be its own ecosystem, and by definition, an Ecosystem is Self-Sustaining.

Thanks for the support. It's funny that you mention TKLBAM reliance on
Amazon because somewhat contrary to common perception (and maybe common
sense) we actually have no financial interest in users storing TKLBAM
data to Amazon S3. Currently, the opposite is true.

TKLBAM is the most popular (by far) service the Hub provides. Somewhat
paradoxically not only are we not making any money from it, it's
actually costing us a bit because of how Amazon's billing system
calculates transaction fees. Every time Amazon sends a user a bill it
charges us 30 cents for the transaction. Even if your bill is 1 cent.
And quite a few users get a 1 cent bill. 30 cents per user isn't a lot,
but it adds up. Thankfully Amazon reduced the storage fees after we
introduced the service by 1 cent per GB so that some of our larger users
are offsetting what it costs us. At least that. But it's still a net
negative for us.

As you might imagine, there have been some internal debates about
charging something for it, at least to offset what it's costing us but
we don't really like that idea because we believe TKLBAM is currently
one of the most important features that attract users to TurnKey and we
want to make it as easy as possible for everyone to try it.

Theoretically if TKLBAM usage scales up far enough Amazon starts giving
TurnKey discounts on storage space and then it might be profitable
without changing anything. But you have to store a hell of a lot of data
to get discounts from Amazon. Maybe in a couple of years...

We want the build process to be available to everyone. We want appliances developed under publically available revision control. We want people to be able to fork Core (or any other appliance), build their own version, choose the build type, get the infrastructure to build it for them, launch it in the cloud if they want, etc.

Personally I love amazon and have been experimenting heavily with it with the view to use it in my business (sell the service).

Imagine telling someone they can run their appliances on site but if the server was to fail a backup could be restored in the cloud within hours (minutes?). While webservers will have DNS issues things like CRM, ERP, file servers etc all benefit from this great integration with elastic computing.

I have zero problem paying for the service knowing the code is open. There are some quirks in some things I would love to sponsor to be resolved however.

The idea of your service is so awesome (huge amount of time saved administering these systems) - why not offer a profitable option to ensure sustainability? Or offer tiered pricing? i.e. low price for up to 10 users, a higher price up to 200 users and a larger price for over 200 users?

i.e. software evaluators need access to the software for a substancial period to complete their checks. So some low-cost version will always be needed just to explore the software.

Also people not coming from a position of ignorance do recognise there are significant costs involved in the infrastructure of developing, storing and distributing OS software such as TKL apps.

From memory (might have to check Australian Bureau of Statistics for accuracy), in Australia most small businesses are less than 200 staff with the majority less than 25 staff - the ones with more than 25 staff can begin to afford competitive fees. Businesses around the 25 staff mark are typically (with a few exceptional businesses) borderline and run tight operations.

Obviously, the community would like some sort of assurance that TKL operations are run as efficiently/effectively as possible (no one wants to pay a premium for lack of efficiency or to be taken advantage of).

Looking forward to the open model and ability for developers to build their own TK Apps based on the core. However, would not like to see TurnKey go under because no one wants to pay for a service that is clearly beneficial (i.e. free-riding on commendable efforts).

I, for one, believed that the prices listed on TKL were profitable ones and was excited that such a great service could be sustainably provided for such an affordable price. Now I'm a little concerned that I might not be able to afford a sustainable price (at the moment I'm interested in exploring the features/capabilities of TKL and form a judgement over its suitability for production use in our environment). Hope that sustainable pricing of these apps is not too much higher :-)

Arguably the best thing about OS is not necessarily that it costs less (because it doesn't always when you look at Total Cost of Ownership), it is that the software can be modified how one wishes to - if/when required.

Sometimes it's important to talk clearly to the community, when I read this post on my cell phone (minutes after it appeared on twitter) I wanted to comment. But I was leaving for a trip, and one week later, I come back to see the comments and it was a surprise! I think the first comment Liraz made IS the post. It was clear, it talked some realities of life and it comes in good timing (emergency is almost finished). The team proved it's commitment to the project, and explained the situation to everyone that cares a bit about TurnkeyLinux. Thanks for that, straight communication is important.

I myself fear of opensource projects without streams of revenue. I skip those, because of my experience with great software out there that was left abbandoned. And Turnkeylinux, being the project which I have invested the most of my time contributing, has a special meaning to me. Besides when I discovered the project, I felt like a dream come true. I have learned a lot from the project, and I've also been critic where I think it deserved some critic.

I'm still waiting for the project to be more open, but I understand you have done the best in your hands to keep things going, so I'll just be patient and hope we can see the fruits for 12.04 to be a great re-launch for the project. A lot of good software had made it's way to the ubuntu official repos, and that also makes it perfect for easy construction of new appliances.

Adrian, you're the best kind of critic: the kind that tells us what we need to hear rather than what we want to. We've privately discussed the problems facing the project a few months back and you played an important role in helping us just how truly critical it was for us to open up as much as possible.

I wish it was possible to just press a button and make it happen immediately. We're working on it, but there seems to be a never-ending stream of things that need to get done and various distractions, not all of them enjoyable. We're working with the resources we have, not the resources we'd like to have.

Once we get some traction going on opening up the project hopefully you and other good people will be able to swoop in and help us make things happen faster. Thanks!

First of all, I am an open source developer (http://www.mltframework.org/) where I also provide professional services around it part-time. So I know what you are dealing with. At first, I was an anonymous user of a TKL appliance, but now I am a customer of the Hub. I expect you to give more consideration to my feedback than some non-paying user, but ultimately I realize it is your decision.

I am building a web service that uses the hub, tklbam, and other Amazon services. Functionally, it is mostly working, but I need to figure out a way to do the e-commerce bits of setting up an account, tracking usage, billing, and accepting payments - much like you did with the Hub atop Amazon. Now, I could either run all customers through one service and bill per time utilized, or I could take advantage of the cloud technology to let each customer spin up their own service based on my custom appliances (using core + tklbam). How can you make the Hub easier to build and run a custom service that simply lets me add my % incremental cost?

We're interested in everyone's feedback, but your feedback is
particularly interesting because it hints at possible future partnership
opportunities. Partnering up with other open source vendors in a way
that is beneficial for both sides seems like something that could be a
win-win for everyone.

I admit, we hadn't previously given serious consideration to this kind
of usage scenario. This is going to need some thought.

We don't have a working solution to offer you right now. In general,
it's really hard to build a platform that another small business (I.e.,
one that doesn't have their own billing infrastructure) can build on
top. You have to get all the pieces in there or it doesn't work. As I've
commented on the Hub pricing thread
Amazon is the only company that seem to get it.

One way to go about this that wouldn't need major changes would be to
add this as a TurnKey appliance to the library like any other, and
include an interface through your web application to enter an API key
that links it with your cloud service. That's not very streamlined
though and it doesn't take care of your billing infrastructure problem
but maybe if the service is compelling enough users would go for it.

Another way would be to leverage Amazon's billing system. It would
require some (but not too much) tweaking on our end. You would create
and maintain your own DevPay AMI based on TurnKey Core or LAMP stack
(for example). You ask users to sign up for it from your website and
then have the subscription pipeline redirect to the Hub on completion.
If the user doesn't have a Hub account they would create one as part of
the process. Then the Hub would present them with a launch page with
your appliance name filled in. Also your AMI would be added to the list
of supported appliances so that they can launch additional appliances of
the same type as long as long as they are subscribed to the DevPay
product. Hmmm... we could probably get something like that to work if
there was enough demand for it.

There another option that may be attractive to you but you'll have to
wait (e.g., a month or two?) for Amazon to make an official announcement
because it's still under wraps and we promised not to talk about it.

I just read an article that was quite relevant to ths discussion. And IMO the fact that it is written by a guy talking from experience working with some software that constitutes a TKL appliance (OTRS) makes it cooler...

Many relevant comments above. I do also appreciate the direct communication from Liraz. I know that my comments are coming in late on this subject. But, I think that's relevant because I am one of the users who goes for short spurts of usage on my TKL projects. Being that user, I admit I've been billed just a few cents a month for well over a year. I didn't realize that I was costing the project money until reading this post. I'd just like to throw it out there that I'd be completely OK if my HUB account was charged a minimum amount that AT LEAST brought it up to a minimum amount that wouldn't cost the project any $$'s. I actually like the idea of supporting TKL through my usage/storage fees. I think that I could go without a $3.60 Latte at some point during the year to avoid putting a financial burden on a project that I truly enjoy working with (and am learning alot from). Just my $.02 to keep my $.02 from causing trouble each month. -Brian