While reading “The Illusion of Asymmetric Insight”, it occurred to me that the real cost (and tragedy) of Unity and GNOME Shell, or KDE and GNOME, is that it turns us into “us” and “them” – and at that point it is really easy to fall into the trap of reducing all criticism to “haters gonna hate”.

The Harmony agreements reached a significant milestone this week, as they were tagged 1.0 and left the “beta” stage. As someone who has previously taken position regarding contributor licensing agreements, I was asked this week what my thoughts on Harmony are.

First off, let me say that I have not followed the Harmony process closely. Indeed, the process, which was semi-open, but operated under Chatham House Rules (any participant can quote what was said in a meeting, but cannot name the person who said it), is one of the major issues I have seen people take with Harmony. The lack of a clearly identified team taking responsibility for the contents and standing behind the agreement texts is unfortunate, but I think it’s an issue completely independent of their content and the project’s goals.

The goal of the project, as far as I can tell, is to provide a set of templates for people who might want to use a Contributor Licensing Agreement (CLA). As far as it goes, that is fine. Where there is a danger is if the existence of such a template is used to encourage the adoption of CLAs (including copyright assignment) as a “best practice” to be followed.

A CLA is actually a conflation of two very different things: the first is asking a contributor to certify that they have the right to make their contribution (that it is original work, that they agree to the project’s license, that their employer has given permission for the contribution, etc). The Mozilla project gets their contributors to sign a similar document upon becoming committers, to ensure that they perform due diligence before accepting a patch from a proposer. So this aspect of CLAs is sensible and useful for most projects.

The second part of CLAs is copyright licensing or assignment. This creates an asymmetric situation in the project where a central copyright holder has the power to make certain decisions for the project, including for some code which they did not write. As I said previously, copyright assignment has its down sides: it will prevent (or, at best, make much more difficult) the formation of a diverse developer community around the core of your project. If that is part of what you want to achieve, then you should be aware of that. However, if you are happy to be basically the sole contributor to the project core, and the ownership of all of the code in that core is useful for other goals, then copyright assignment may well be appropriate for your project.

Harmony does attempt to make copyright assignment more acceptable by including a licensing promise in some variants. This is fine. I don’t think I would sign such an agreement, but I am sure that the promise that contributions will always be available under a certain license may be enough to reassure other potential contributors.

The main “flaw” which others have identified in Harmony is the lack of a patent promise from the assignee to the assigner. I kind of think that this is a red herring, because such a promise should really be explicit in the license under which the contributor got the software in the first place. Having such a promise in a CLA really doesn’t feel necessary or useful.

Overall, I’m sure that some people will find Harmony agreements useful – they will hopefully save communication time between projects with CLAs and developers, and lawyer fees for companies considering the adoption of a CLA. Yet, my priority will continue to be to question the assumptions which lead people to adopt a CLA without fully thinking through the consequences.

Do you really need a CLA to achieve your objectives? Is it, in fact, harmful to some of what you want to achieve? At the end of the day, my position remains the same: the goal should not be to write a better CLA, it should be to figure out whether we can avoid one altogether, and figure out how to create and thrive in a vibrant developer community.

Technology can make it easier to collaborate when applications mirror a more intuitive, fluid work style, but selecting a tool without addressing roles, processes, metrics and the organization’s workplace climate is putting the cart before the horse.

I’ve been thinking a lot recently about mentoring programs, what works, what doesn’t, and what the minimum amount of effort needed to bootstrap a program might be.

With the advent of Google Summer of Code and Google Code-In, more and more projects are formalising mentoring and thinking about what newcomers to the project might be able to do to learn the ropes and integrate themselves into the community. These programs led to other organised programs like GNOME’s Women Summer Outreach Program. Of course, these initiatives weren’t the first to encourage good mentoring, but they have helped make the idea of mentors taking new developers under their wing much more widespread.

In addition to these scheduled and time-constrained programs, many projects have more informal “always-on” mentoring programs – Drupal Dojo and GNOME Love come to mind. Others save smaller easier tasks for newcomers to cut their teeth on, like the “easy hacks” in LibreOffice. Esther Schindler wrote a fantastic article a few years ago documenting best and worst practices in mentoring among different projects.

Most mentoring programs I have seen and participated in don’t have a very good success rate, though. In this article, I look at why that is the case, and what can be put in place to increase the success rate when recruiting new developers.

Why most mentoring fails

Graham Percival, a GNU/LilyPond developer, decided in 2008 to run an experiment. At one point, Graham decided that he would quit the project, but felt guilty about doing so in one go. So he started the “Great Documentation Project” to recruit a replacement documentation team to follow on after his departure. He then spent 12 months doing nothing but mentoring newcomers to get them involved in the project, and documented his results. Over the course of a year, he estimates that he spent around 800 hours mentoring newcomers to the project.

His conclusions? The net result for the project was somewhere between 600-900 hours of productivity, and at the end of the year, 0 new documentation team members. In other words, Graham would have been better off doing everything himself.

Graham found that “Only 1 in 4 developers was a net gain for the project” – that is, for every 4 apprentices that Graham spent time mentoring, only 1 hung around long enough for the project to recoup the time investment he put into mentoring. A further 1 in 4 were neither gain or loss – their contribution roughly equalled the mentor time that they took up. And the remainder were a net loss for the project, with much more time spent mentoring than the results could justify.

The GNOME Women’s Summer Outreach Program in 2006 had 6 participants. In 2009, the GNOME Journal ran a “Where are they now?” follow-up article. Of the 6 original participants, only one is still involved in free software, and none are involved in GNOME. Murray Stokely did a follow-up in 2008 to track the 57 alumni of Summer of Code who had worked on FreeBSD. Of these, 10 students went on to get full commit access, and a further 4 students were still contributing to FreeBSD or OpenBSD after the project. Obey Arthur Liu also did a review of Debian participants in 2008. Of 11 students from 2008 who had no previous Debian developer experience, he found that 4 remained active in the project one year later.

From my own experience as a replacement mentor and administrator in the Summer of Code for the GIMP in 2006, we had 6 projects, most of which were considered a success by the end of the summer, yet of the participating students, none have made any meaningful contribution to the GIMP since.

I feel safe in saying that the majority of mentoring projects fail – and Graham’s 1 in 4 sounds about right as an overall average success/failure rate. This begs the question: why?

Most mentored projects take too long

What might take a mentor a couple of hours working on his own could well take an apprentice several days or weeks. All of the experience that allows you to hit the ground running isn’t there. The most important part of the mentoring experience is getting the student to the point where he can start working on the problem. To help address this point, many projects now require Summer of Code applicants to compile the project and propose a trivial patch before they are accepted for the program, but understanding the architecture of a project and reading enough code to get a handle on coding conventions will take time. It will also take mentor time. It takes longer to teach a newcomer to your project than to do the work yourself, as anyone who has ever had a Summer intern will attest.

When you set a trainee task which you estimate to be about 4 hours work, that will end up costing a few weeks of volunteer effort for your apprentice, and 8 to 10 hours mentoring time for you during that time. Obviously, this is a big investment on both sides, and can lead to the apprentice giving up, or the mentor running out of patience. I remember in the first year of Summer of Code, projects were taking features off their wishlists that had not been touched for years, and expecting students new to the project to come in and work full time implementing them perfectly over the course of 12 weeks. The reality that first year was that most of the time was spent getting a working environment set up, and getting started on their task.

Mentoring demands a lot of mentors

As a free software developer, you might not have a lot of time to work on your project. Josh Berkus, quoted in Schindler’s article, says “being a good mentor requires a lot of time, frequently more time than it would take you to do the development yourself”. According to the Google Summer of Code FAQ, “5 hours a week is a reasonable estimate” for the amount of time you would need to dedicate to mentoring. Federico Mena Quintero suggests that you will need to set aside “between 30 and 60 minutes a day“.

When you only have 10 hours a week to contribute to a project, giving up half of it to help someone else is a lot. It is easy to see how working on code can get a higher priority than checking in with your apprentice to make sure everything is on track.

Communication issues

More mentoring projects fail for lack of communication than for any other reason.

Apprentices may expect their mentors to check in on them, while mentors expect apprentices to ask questions if they have any. Perhaps newcomers to the project are not used to working on mailing lists, or are afraid of asking stupid questions, preferring to read lots of documentation or search Google for answers. In the absence of clear guidelines on when and how parties will talk to each other, communication will tend towards “not enough” rather than “too much”.

No follow through

Many mentoring programs stop when your first task is complete. The relationship between the mentor and the apprentice lasts until the end of the task, and then either the apprentice goes off and starts a new task, with a new mentor, or that is the end of their relationship with the project. I would be really interested to hear how many Summer of Code mentors maintained a relationship with their students after the end of the Summer, and helped them out with further projects. I suspect that many mentors invest a lot of time during the program, and then spend most of their time catching up with what they wanted to do.

Project culture

In her OSCON keynote in 2009, Skud talked about the creation of a welcoming and diverse community as a prerequisite for recruiting new developers. Sometimes, your project culture just doesn’t match newcomers to the project. If this happens regularly, then perhaps the project’s leaders need to work on changing the culture, but this is easier said than done. As Chris di Bona says in this video, “the brutality of open source is such that people will learn to work with others, or they will fail”. While many think that this kind of trial-by-fire is fine, the will not be the environment for everyone. It is really up to each project and its leaders to decide how “brutal” or forgiving they want to be. There is a trade-off: investing time in apprentices who will contribute little is a waste of time, but being too dismissive of a potential new developer could cost your project in the long run.

Mentoring best practices

Is all the effort worth it? If mentoring programs are so much hassle, why go to the bother?

Mentoring programs are needed to ensure that your project is long-term sustainable. As Graham says in his presentation: “Core developers do most of the work. Losing core developers is bad. Projects will lose core developers.” Do you need any other reason to start actively recruiting new blood?

There are a few simple things that you can put in place to give your mentoring program a better chance of success.

Small tasks

Mentored tasks should be small, bite-sized, and allow the apprentice to succeed or fail fast. This has a number of advantages: The apprentice who won’t stick around, or who will accomplish nothing, has not wasted a lot of your mentor’s time. The apprentice who will stay around gets a quick win, gets his name in the ChangeLog, and gains assurance in his ability to contribute. And the quick feedback loop is incredibly rewarding for the mentor, who sees his apprentice attack new tasks and increase his productivity in short order. Graham implies that a 10 minute task is the right size, with the expectation that the apprentice might take 1 hour to accomplish the task.

A ten minute task might even take longer to identify and list than it would to do. You can consider this cost the boot-strapping cost of the mentoring program. Some tasks that are well suited to this might include:

Write user documentation for 1 feature

Get the source code, compile it, remove a compiler warning, and submit a patch

Critique 1 unreviewed patch in Bugzilla

Fix a trivial bug (a one line/local change)

Of course, the types of tasks on your list will change from one project to the next.

Mentoring is management

Just as not everyone is suited to being a manager, not everyone is suited to being a mentor. The skills needed to be a good mentor are also those of a good manager – keeping track of someone else’s activity while making them feel responsible for what they’re working on, communicating well and frequently, and reading between the lines to diagnose problems before it’s too late to do anything about them.

When you think of it in this way, there is an opportunity for developers who would like to gain management experience to do so as a mentor in a free software project. Continually recruiting mentors is just as important as recruiting developers. Since mentoring takes a lot of time, it’s important that mentors get time off and new mentors are coming in in their place.

Pair apprentices with mentors, not tasks

An apprentice should have the same mentor from the day he enters the mentoring program until he no longer needs or wants the help. The relationship will ideally continue until the apprentice has himself become a mentor. Free software communities are built on relationships, and the key point to a mentoring program is to help the creation of a new relationship. Mentoring relationships can be limited in time also, 6 months or a year seem like good time limits. The time needed to mentor will, hopefully, go down over this period.

Regular meeting times

Mentors and apprentices should ensure that there is a time on their calendar for a “one on one” at regular times. How regularly will depend on the tasks, and the amount of time you can spend on it. Weekly, fortnightly or monthly are all reasonable in different situations. This meeting should be independent of any other communication you have with the person – it is too easy for the general business of a project to swallow up a newbie and prevent their voice from being heard. Rands said it well when he said “this chatter will bury the individual voice unless someone pays attention.”

Convert apprentices into mentors

Never do you understand the pain of the initial learning curve better than when you have just gone through it. The people best suited to helping out newcomers to the project are those who have just come through the mentoring program themselves.

This is a phenomenon that I have seen in the Summer of Code. Those students who succeed and stay with the project are often eager to become mentors the following year. And they will, in general, be among the best mentors in the project.

Keep track

For all involved, it’s useful to have some idea of the issues newcomers have – ensure that documenting solutions is part of what you ask. It’s also useful to know how successful your mentoring program is. Can you do better than the 1 in 4 success rate of LilyPond? Keeping track of successes and failures encourages new mentors, and gives you data to address any problems you run into.

Manage the mentors

All of this work has overhead. In a small project with 1 or 2 core developers, it’s easy enough to have each core developer take an apprentice under their wing, and co-ordinate on the mailing list. In bigger projects, keeping track of who is a mentor, and who is mentoring who, and inviting new mentors, and ensuring that no-one falls through the cracks when a mentor gets too busy, is a job of itself. If your mentoring program goes beyond more than ~5 mentors or so, you might want to consider nominating someone to lead the program (or see who steps up to do the job). This is the idea behind the Summer of Code administrator, and it’s a good one.

Go forth and multiply

Developer attrition is a problem in open source, and recruitment and training of new developers is the only solution. Any project which is not bringing new developers up to positions where they can take over maintainership is doomed to failure. A good mentoring program, however, with a retention rate around 25%, organised continuously, should ensure that your project continues to grow and attract new developers.

Replenishing your stock of mentor tasks and recruiting new mentors will take effort, and continual maintenance of someone putting in a few hours a week. If you execute well, then you will have helped contribute to the long term diversity and health of your project.

If you’re contributing a one line patch to a big corpus of code, there’s a good argument that this is insufficient to grant you any kind of authority in the project.

But how about if you’re contributing a major feature? Surely you get some say in how your code evolves over time? And if you’re a company, contributing thousands of man-hours and dozens of features to the project, isn’t it reasonable that you get a say in all the decisions related to the project, including licensing decisions?

Let’s take the analogy of the gift of the plant & run with it.

If I offer you a potted plant, I have no reasonable expectations of you. I can’t even tell you where to plant it. It’s yours, the house is yours, my involvement is finished.

But let’s say we start a relationship, and it starts to get serious. I start to sleep over some nights, maybe leave a toothbrush in your bathroom. Do I now have some say about what happens in the house? Probably not about anything important, but you might solicit my opinion for any furniture purchases, it might be OK for me to tidy up once in a while.

Now, things get really serious, and I move in. It’s still your house, but surely I get a say now in everything. Of course, everything that was there before I arrived is yours, maybe you keep the game room just the way you want it. But we discuss and reach an agreement for everything from decorating decisions to which cable supplier we’re going to get. We’re building something shared. Sure, I moved into your house, but now it’s our home. If you decide to sell it, there’s not much I can do, but if you did it without talking to me, I’d be really pissed off. It would signal the end of our relationship, probably.

Let’s go one step further. We get married. We commit to sharing our lives. Surely I get a say in whether you sell the house now?

Unless…

Unless when I offered you that plant on our first date, you asked me to sign an agreement, saying that what was yours was yours, and any future improvements we might make to the house together would be yours too, and of course I could not exercise a claim of ownership over the house.

You never know, I might sign it. I might even offer you a second plant for the house. If you date a lot, you might get a lot of plants. I don’t know if I’d spend money to help you renovate the kitchen, though.

One of the most important things you can do in a free software project, besides writing code, is to get your key contributors together as often as possible.

I’ve been fortunate to be able to organise a number of events in the past 10 years, and also to observe others and learn from them over that time. Here are some of the lessons I’ve learned over the years from that experience:

Venue

The starting point for most meetings or conferences is the venue. If you’re getting a small group (under 10 people) together, then it is usually OK just to pick a city, and ask a friend who runs a business or is a college professor to book a room for you. Or use a co-working space. Or hang out in someone’s house, and camp in the garden. Once you get bigger, you may need to go through a more formal process.

If you’re not careful, the venue will be a huge expense, and you’ll have to find that money somewhere. But if you are smart, you can manage a free venue quite easily.

Here are a few strategies you might want to try:

Piggy-back on another event – the Linux Foundation Collaboration Summit, OSCON, LinuxTag, GUADEC and many other conferences are happy to host workshops or meet-ups for smaller groups. The GIMP Developers Conference in 2004 was the first meet-up that I organised, and to avoid the hassle of dealing with a venue, finding a time that suited everyone, and so on, I asked the GNOME Foundation if they wouldn’t mind setting aside some space for us at GUADEC – and they said yes.Take advantage of the bigger conference’s organisation, and you get the added benefit of attending the bigger conference at the same time!

Ask local universities for free rooms – This won’t work once you go over a certain size, but especially for universities which have academics who are members of the local LUG, they can talk their department head into booking a lecture theatre & a few classrooms for a weekend. Many universities will ask to do a press release and get credit on the conference web-site, and this is a completely fair deal.The first Libre Graphics Meeting was hosted free in CPE Lyon, and the GNOME Boston Summit has been hosted free for a number of years in MIT.

If the venue can’t be free, see if you can get someone else to pay for it – Once your conference is bigger than about 200 people, most venues will require payment. Hosting a conference will cost them a lot, and it’s a big part of the business model of universities to host conferences when the students are gone. But just because the university or conference center won’t host you for free doesn’t mean that you have to be the one paying.

Local regional governments like to be involved with big events in their region. GUADEC in Stuttgart, the Gran Canaria Desktop Summit, and this year’s Desktop Summit in Berlin have all had the cost of the venue covered by the host region. An additional benefit of partnering with the region is that they will often have links to local industry and press – resources you can use to get publicity and perhaps even sponsorship for your conference.

Run a bidding process – by encouraging groups wishing to host the conference to put in bids, you are also encouraging them to source a venue and talk to local partners before you decide where to go. You are also putting cities in competition with each other, and like olympic bids, cities don’t like to lose competitions they’re in!

Budget

Conferences cost money. Major costs for a small meet-up might be
covering the travel costs of attendees. For a larger conference, the
major costs will be equipment, staff and venue.

Every time I have been raising the budget for a conference, my rule of
thumb has been simple:

Decide how much money you need to put on the event

Fundraise until you reach that amount

Stop fundraising, and move on to other things.

Raising money is a tricky thing to do. You can literally spend all of
your time doing it. At the end of the day, you have a conference to put
on, and the amount of money in the budget is not the major concern of
your attendees.

Remember, your primary goal is to get project participants together to
advance the project. So getting the word out to prospective attendees,
organising accommodation, venue, talks, food and drinks, social
activities and everything else people expect at an event is more
important than raising money.

Of course, you need money to be able to do all the rest of that stuff,
so finding sponsors, fixing sponsorship levels, and selling your
conference is a necessary evil. But once you have reached the amount of
money you need for the conference, you really do have better things to
do with your time.

There are a few potential sources of funds to put on a conference – I
recommend a mix of all of these as the best way to raise your budget.

Attendees – While this is a controversial topic among many communities, I think it is completely valid to ask attendees to contribute something to the costs of the conference. Attendees benefit from the facilities, the social events, and gain value from the conference.Some communities consider attendance at their annual event as a kind of reward for services rendered, or an incitement to do good work in the coming year, but I don’t think that’s a healthy way to look at it.

There are a few ways for conference attendees to fund the running of the conference:

Registration fees – This is the most common way to get money from conference attendees. Most community conferences ask for a token amount of fees. I’ve seen conferences ask for an entrance fee of €20 to €50, and most people have not had a problem paying this.

A pre-paid fee also has an additional benefit of massively reducing no-shows among locals. People place more value on attending an event that costs them €10 than one where they can get in for free, even if the content is the same.

Donations – very successfully employed by FOSDEM. Attendees are offered an array of goodies, provided by sponsors (books, magazine subscriptions, t-shirts) in return for a donation. But those who want can attend for free.

Selling merchandising – Perhaps your community would be happier hosting a free conference, and selling plush toys, t-shirts, hoodies, mugs and other merchandising to make some money. Beware: in my experience you can expect less from profits from merchandising sales than you would get giving a free t-shirt to each attendee with a registration fee.

Sponsors – Media publications will typically agree to “press sponsorship” – providing free ads for your conference in their print magazine or website. If your conference is a registered non-profit which can accept tax-deductible donations, offer press sponsors the chance to invoice you for the services and then make a separate sponsorship grant to cover the bill. The end result for you is identical, but it will allow the publication to write off the space they donate to you for tax.

What you really want, though, are cash sponsorships. As the number of free software projects and conferences has multiplied, the competition for sponsorship dollars has really heated up in recent years. To maximise your chances of making your budget target, there are a few things you can do.

Conference brochure – Think of your conference as a product you’re selling. What does it stand for, how much attention does it get, how important is it to you, to your members, to the industry and beyond? What is the value proposition for the sponsor?

You can sell a sponsorship package on three or four different grounds: perhaps conference attendees are a high-value target audience for the sponsor, perhaps (especially for smaller conferences) the attendees aren’t what’s important, it’s the attention that the conference will get in the international press, or perhaps you are pitching to the company that the conference is improving a piece of software that they depend on.

Depending on the positioning of the conference, you can then make a list of potential sponsors. You should have a sponsorship brochure that you can send them, which will contain a description of the conference, a sales pitch explaining why it’s interesting for the company to sponsor it, potentially press clippings or quotes from past attendees saying how great the conference is, and finally the amount of money you’re looking for.

Sponsorship levels – These should be fixed based on the amount of money you want to raise. You should figure on your biggest sponsor providing somewhere between 30% and 40% of your total conference budget for a smaller conference. If you’re lucky, and your conference gets a lot of sponsors, that might be as low as 20%. Figure on a third as a ball-park figure. That means if you’ve decided that you need €60,000 then you should set your cornerstone sponsor level at €20,000, and all the other levels in consequence (say, €12,000 for the second level and €6,000 for third level).

For smaller conferences and meet-ups, the fundraising process might be slightly more informal, but you should still think of the entire process as a sales pitch.

Calendar – Most companies have either a yearly or half-yearly budget cycle. If you get your submission into the right person at the right time, then you could potentially have a much easier conversation. The best time to submit proposals for sponsorship of a conference in the Summer is around October or November of the year before, when companies are finalising their annual budget.

If you miss this window, all is not lost, but any sponsorship you get will be coming out of discretionary budgets, which tend to get spread quite thin, and are guarded preciously by their owners. Alternatively, you might get a commitment to sponsor your July conference in May, at the end of the first half budget process – which is quite late in the day.

Approaching the right people – I’m not going to teach anyone sales, but my personal secret to dealing with big organisations is to make friends with people inside the organisations, and try to get a feel for where the budget might come from for my event. Your friend will probably not be the person controlling the budget, but getting him or her on board is your opportunity to have an advocate inside the organisation, working to put your proposal in front of the eyes of the person who owns the budget.

Big organisations can be a hard nut to crack, but free software projects often have friends in high places. If you have seen the CTO or CEO of a Fortune 500 company talk about your project in a news article, don’t hesitate to drop him a line mentioning that, and when the time comes to fund that conference, a personal note asking who the best person to talk to will work wonders. Remember, your goal is not to sell to your personal contact, it is to turn her into an advocate to your cause inside the organisation, and create the opportunity to sell the conference to the budget owner later.

Also, remember when you’re selling sponsorship packages that everything which costs you money could potentially be part of a sponsorship package. Some companies will offer lanyards for attendees, or offer to pay for a coffee break, or ice-cream in the afternoon, or a social event. These are potentially valuable sponsorship opportunities and you should be clear in your brochure about everything that’s happening, and spec out a provisional budget for each of these events when you’re drafting your budget.

Content

Conference content is the most important thing about a conference. Different events handle content differently – some events invite a large proportion of their speakers, while others like GUADEC and OSCON invite proposals and choose talks to fill the spots.

The strategy you choose will depend largely on the nature of the event. If it’s an event in its 10th year with an ever increasing number of attendees, then a call for papers is great. If you’re in your first year, and people really don’t know what to make of the event, then setting the tone by inviting a number of speakers will do a great job of helping people know what you’re aiming for.

For Ignite Lyon last year, I invited about 40% of the speakers for the first night (and often had to hassle them to put in a submission, and the remaining 60% came through a submission form. For the first Libre Graphics Meeting, apart from lightning talks, I think that I contacted every speaker except 2 first. Now that the event is in its 6th year, there is a call for proposals process which works quite well.

Schedule

Avoiding putting talks in parallel which will appeal to the same people is hard. Every single conference, you hear from people who wanted to attend talks which were on at the same time on similar topics.

My solution to conference scheduling is very low-tech, but works for me. Coloured post-its, with a different colour for each theme, and an empty talks grid, do the job fine. Write the talk titles one per post-it, add any constraints you have for the speaker, and then fill in the grid.

Taking scheduling off the computer and into real life makes it really easy to see when you have clashes, to swap talks as often as you like, and then to commit it to a web page when you’re happy with it.

Parties

Parties are a trade-off. You want everyone to have fun, and hanging out is a huge part of attending a conference. But morning attendance suffers after a party. Pity the poor community member who has to drag himself out of bed after 3 hours sleep to go and talk to 4 people at 9am after the party.

Some conferences have too many parties. It’s great to have the opportunity to get drunk with friends every night. But it’s not great to actually get drunk with friends every night. Remember the goal of the conference: you want to encourage the advancement of your project.

I encourage one biggish party, and one other smallish party, over the course of the week. Outside of that, people will still get together, and have a good time, but it’ll be on their dime, and that will keep everyone reasonable.

With a little imagination, you can come up with events that don’t involved loud music and alcohol. Other types of social event can work just as well, and be even more fun.

At GUADEC we have had a football tournament for the last number of years. During the OpenWengo Summit in 2007, we brought people on a boat ride on the Seine and we went on a classic 19th century merry-go-round afterwards. Getting people eating together is another great way to create closer ties – I have very fond memories of group dinners at a number of conferences. At the annual KDE conference Akademy, there is typically a Big Day Out, where people get together for a picnic, some light outdoors activity, a boat ride, some sightseeing or something similar.

Extra costs

Watch out for those unforeseen costs! One conference I was involved in, where the venue was “100% sponsored” left us with a €20,000 bill for labour and equipment costs. Yes, the venue had been sponsored, but setting up tables and chairs, and equipment rental of whiteboards, overhead projectors and so on, had not. At the end of the day, I estimate that we used about 60% of the equipment we paid for.

Conference venues are hugely expensive for everything they provide. Coffee breaks can cost up to $10 per person for a coffee & a few biscuits, bottled water for speakers costs $5 per bottle, and so on. Rental of an overhead projector and mics for one room for one day can cost €300 or more, depending on whether the venue insists that equipment be operated by their a/v guy or not.

When you’re dealing with a commercial venue, be clear up-front about what you’re paying for.

On-site details

I like conferences that take care of the little details. As a speaker, I like it when someone contacts me before the conference and says they’ll be presenting me, what would I like them to say? It’s reassuring to know that when I arrive there will be a hands-free mic and someone who can help fit it.

Taking care of all of these details needs a gaggle of volunteers, and it needs someone organising them beforehand and during the event. Spend a lot of time talking to the local staff, especially the audio/visual engineers.

In one conference, the a/v guy would switch manually to a screen-saver at the end of a presentation. We had a comical situation during a lightning talk session where after the first speaker, I switched presentations, and while the next presentation showed up on my laptop, we still had the screensaver on the big screen. No-one had talked to the A/V engineer to explain to him the format of the presentation!

So we ended up with 4 Linux engineers looking at the laptop, checking connections and running various Xrandr incantations, trying to get the overhead projector working again! We eventually changed laptops, and the a/v engineer realised what the session was, and all went well after that – most of the people involved ended up blaming my laptop.

Have fun!

Running a conference, or even a smaller meet-up, is time consuming, and consists of a lot of detail work, much of which will never be noticed by attendees. I haven’t even dealt with things like banners and posters, graphic design, dealing with the press, or any of the other joys that come from organising a conference.

The end result is massively rewarding, though. A study I did last year of the GNOME project showed that there is a massive project-wide boost in productivity just after our annual conference, and many of our community members cite the conference as the high point of their year.

After my rather controversial question a few days ago and multiple reactions from around the KDE & Canonical world, a lot of reading and digging into archives, and a lot of conversations with people across the spectrum, I have some preliminary findings and lessons which I hope can serve us going forward to help improve things. Nothing in here is controversial, I think, but each of these is a contributing factor to the current mess we find ourselves in.

tl;dr

For those without the patience to read this article (which is much longer than I intended it to be when I started!), here are the headline points:

FreeDesktop.org is broken as a standards body

Mark Shuttleworth doesn’t understand how GNOME works

GNOME is not easy to understand

Deep mistrust has developed between Canonical, GNOME & KDE

Difficult people are prominent in each of these projects

Behind closed doors conversations are poison

For people to work together, they need to be in the same place

In summary, there are a number of things we can do to move forward from where we are now: improve processes & structure for freedesktop.org (this will require buy-in from key GNOME & KDE people), make the operation of GNOME (and the operation of individual modules) more transparent from outside the project, cut out a lot of the back-channel conversations that have been happening over the phone, in person & on IRC, in favour of documented & archived discussions and agreements on mailing lists & wikis, and work to ensure that people working on similar problem areas are talking to each other.

The major challenge we have is how to move beyond the deep mistrust which has evolved between members of our communities, who are all to eager to assume the worst of others, and how we can improve the tone of discourse when some of the most prominent members of our communities are also hard to work with.

This is not surprising when you consider that it’s written right there on the front page: “freedesktop.org is not a formal standards organization”.

In the case of the StatusNotifier spec, the brokenness shines through. Work started in April 2009 by Aaron Seigo, using the Galago spec as a starting point. Once KDE had begun working on an implementation, Marco Martin started on an initial draft of a spec. The first round draft there was mostly done on September 17 and proposed as the KNotificationItem spec. Then Aurélien Gâteau and Ted Gould made some (offline) suggestions, resulting in a rename, and some revisions, in late October. The spec was proposed as the StatusNotifier spec in December 2009.

At the point that GNOME developers Dan Winship & Mathias Clasen, and Citrix developer Giles Atkinson, reviewed the spec and made comments on it, too much had been invested in it to make major revisions. At that point, it is disingenuous to call StatusNotifier a cross-desktop standard. Hosting a document on the freedesktop.org wiki does not a cross-desktop standard make.

instead, the idea was, “If we propose it on fd.o, then people have to accept it because otherwise they won’t be cooperating with fd.o.” this is completely different from trying to work with others and having those efforts ignored.

In fact, that is exactly how StatusNotifiers were perceived (and exactly how Mark & Aaron are messaging GNOME’s non-adoption of the spec).

There is no freedesktop.org process for proposing standards, identifying those which are proposals and those which are de facto implemented, and perhaps more importantly, there is no process for building consensus around a specification, and signalling that consensus.

If I were in charge, I would require every spec to start with a problem definition. Only by agreeing on the problem can we hope to arrive at a solution which will be acceptable to all. The problem statement is the guiding light of a spec. Then I would make sure that the people with an interest in solving the problem were committed to the project. Only then do you start working on a spec and implementations. Without agreement on a problem, and without the right people at the table from the start, the effort is doomed. Some guidance on the process for the creation of a spec would be a start.

In this case, there was no founding problem statement. The spec proposed by Marco Martin listed this as the problems which it was solving:

The new protocol is based upon D-Bus, and separates the presentation of the items from the logic, in our case the painting is completely controlled by Plasma and the applications registers via D-bus (with a small client library shared across KDE) to a central server, while there can be zero or more instances of the systemtray. if either the serve or no instances of systemtrays that supports this protocol are registered the system will fall back using the old freedesktop.org systray specification.

This is not a compelling problem statement. No user ever had a problem because notifications didn’t use D-Bus.

It’s clear when reading Dan Winship’s follow-up comments that there was disagreement on the problem to solve, as well as disagreement on how to solve it. Dan felt that a spec should include policy, and document expected behaviour, while Aaron and Marco were at that point committed to the separation of “the visualisation” and the API. With a better problem statement, this would have been a minor implementation point, without one, two people ended up arguing over positions, and not interests.

If, instead, people had agreed on the problem of the panel or the issues they wanted to address before starting to work on a solution, things might have gone more smoothly. Note that the wiki page linked above was created at the end of December 2009, and the mailing list post was from February 2010 – to communicate what had already been written, not to concentrate people on a common problem.

Mark Shuttleworth really doesn’t know how GNOME works

This one really surprised me, but I think it’s indisputable. Mark wants GNOME to have “strong, mature technical leadership”. He talks about a GNOME cabal, and GNOME’s strategy being “whatever Jon McCann wants to do with the panel”. Mark and others don’t understand why libappindicator was rejected as an external dependency, misunderstanding that external dependencies are, by definition, dependencies of GNOME modules. He admits himself that he has failed to have Canonical developments considered as “internal to Gnome”, and clearly does not understand the position that the GNOME community as a whole has taken with respect to copyright assignment, or the history behind that position.

My understanding of GNOME is this: GNOME does not have technical leadership – it hasn’t had clear technical leadership since, as I understand it, the creation of the GNOME Foundation (at which point, by design, the board was given a mandate to build and define GNOME, and then soon afterwards removed that mandate from itself). The foundation does not now dictate any vision or direction for GNOME.

It can be argued that this is something which should be changed. That change will be effected by people involved in the foundation and the project. It is not enough for Mark to tell the project that “you need leadership”, or Jono Bacon telling foundation members (as he told me in 2007) that they should step up to the plate. Decisions are made by those who turn up – and I consider Mark, Jono, Ted Gould and others as members of the GNOME project, with as much mandate to change GNOME as I do. If Mark wants GNOME to have strong leadership, then he needs to help make that happen.

Given that this is not (yet) how GNOME works, to get things done in GNOME, you need to talk to the right people. That means, defining your problem, and identifying the stakeholders who are also interested in that problem, and working out a solution with them (am I repeating myself?). Mark seems to want GNOME to behave like a company, so that he can get “his people” to talk to “our people” and make it happen. I think that this misunderstanding of how to wield influence within the GNOME project is a key problem.

But then again, over the years I have heard similar feedback from GNOME Mobile participants, and people in Nokia – so it’s not all Mark’s fault. As Jono says here: GNOME does have a reputation of being hard to work with for companies – no point in denying it (then again, so does the kernel, and they seem to get along fine).

GNOME is not easy to understand

When I evaluated GNOME’s governance for Simon Phipps recently, I scored the project 0 (on a scale of -1 to 1) for the criteria of oligarchic governance. The notes from the evaluation were:

Newcomers to GNOME often have trouble figuring out who’s in charge. The Release Team is responsible primarily for the release process and has not traditionally set any strategic direction for GNOME, and individual module governance rules are varied. The foundation board is responsible primarily for maintaining the infrastructure of the foundation, and dealing with sponsors and benefactors, and does not set any technical direction.

Score: Governance is open, membership of the release team oligarchy is meritocratic – scoring zero for oligarchy because much of the governance is devolved to maintainers, making it hard to figure out how to accomplish project-wide change.

Finding the right person inside the GNOME project to help work on a given problem is not straightforward. If you want to make a change to one module, then it’s as simple as working with the maintainers. If, on the other hand, you want to propose a system-wide change, it is a much harder job – you need to work with module maintainers to get them to adopt your proposal, then work with the release team & the wider GNOME community through the module proposal period to get your library included in one of the module sets. Libraries I can think of in recent times that have not gained sufficient traction include Beagle, Geoclue, Soylent, or LeafTag. Other projects like Pyro, GNOME Online Desktop, or Zeitgeist have had baptisms of fire. Even libraries like GStreamer and Telepathy have taken a long time to get traction in core GNOME applications.

Even once you’re in the right place, having work reviewed can take time & effort. I have been told stories of dropped or unreviewed patch-sets by developers I’ve known across a number of projects for many years – one that is mentioned frequently is GNOME Control Center. Maybe persistence was all that would be required, maybe the patches were submitted in a way other than the usual method, or maybe the maintainer was just stuck for time & forgot – in any case, patches were lost, or their integration delayed, and contributors ended up disenfranchised.

But then you can say the same thing about the Linux kernel – contributing to the project is so confusing that Jon Corbet wrote a book about how to contribute, or even KDE – Stuart Jarvis wrote a timely article yesterday explaining that “KDE is not like [a company]. We don’t have leaders. We have prominent community members, but they tend to operate within their own areas of expertise.” Sounds familiar.

The bottom line is that GNOME can improve, but it is not going to change its nature, and working with GNOME needs to be done on the terms of the community you’re working with, not on your terms.

Deep mistrust has developed between Canonical, GNOME and KDE

Regardless of the causes & the history, it’s been made very clear over the past two days that people in our communities are prepared to believe the worst about their fellow free software developers.

@Tom: “do you think the Gnome my way or the high way attitude is connected to company agenda?”

i don’t think so. […] it really seems to be something common to the culture of the project rather than the culture of the companies they work for.

and later:

it’s not a belief that GNOME has decided to not collaborate on this (and other) initiatives for no good reason: it’s a fact. there is a demonstrated “if isn’t invented here, it isn’t used here” pattern of behavior.

Mark clearly believes that GNOME Shell is a Red Hat project. He feels short-changed, feels he and his team have made a good faith effort to engage which has been rejected, offer suggestions which have been ignored. On the other hand, Jon McCann does not see things the same way. And a lot of GNOME people see the move to Unity as a deliberate effort to undermine GNOME Shell, one more in a series of initiatives designed to give Ubuntu differentiation over their competitors without feeding the results into the upstream ecosystem.

Looking at some of the tweets & comments on the various posts, I see an employee of Intel, a developer from the Junta de Andalucia, a number of ex Canonical employees, a Novell employee, an unaffiliated volunteer, and others.Mark’s article blaming GNOME for the problems in the relationship was literally met with “WTF”s and laughter.

Ill will toward Canonical as a company is not limited to GNOME – Greg Kroah Hartman’s infamous presentation at the Linux Plumber’s Conference in 2008 comes to mind. Clearly frustration has been building across the community for a number of years, and it’s far too easy to dismiss it as jealousy because Ubuntu has so many users.

Difficult people are prominent in each of these projects

At this point, the participants in what has become a menage à trois each have a world view which is so different and a prioris so ingrained about the motivations & attitude of everyone associated with another project that undoing the damage will be very difficult.

It’s made even more difficult because a number of key contributors in the projects in question have a reputation of being hard to work with. In GNOME we have our share of people who, to use a phrase Jon Corbet uses to describe kernel hackers, are “not always concerned with showing a high degree of politeness”. I could come up with 10 names of hackers who might themselves be surprised to be on the list, who would be considered by people who have worked with them to be “prickly” to say the least. These people can be found at all levels of the project – prominent on the GNOME Foundation mailing list, maintainers of modules, employees of all of the major companies working with GNOME, even on the release team.

On the KDE side, Aaron also has this reputation – one KDE contributor I spoke to recently said that if Aaron had been a little more open to the feedback he received rather than adopting his “habitual air of superiority” that things might have gone better. And he’s not alone.

Reading the thread on freedesktop.org where the StatusNotifier spec was being discussed, it’s clear that Dan and Matthias considered that Aaron was being dismissive of their concerns – and I can certainly see why. Aaron, on the other hand, in his blog post, considers that “there was a lot of communication about Status Notifiers on the freedesktop.org xdg list where good feedback was offered and the specification improved significantly as a result. So communication really can’t be to blame here, at least not communication by those outside of GNOME” – there is a clear disconnect between how the thread was perceived by Aaron and by other participants.

Mark himself is no angel – I’m sure he will recognise that he is not one to avoid polarising positionn, or to change his mind easily once it’s made up. On the subject of copyright assignment, his mind is made up, for example. There is no revenue motive behind the decision (I am convinced of this), but on principle, Mark has come to believe that controlling the copyrights to code is a best practice, and nothing will change his mind about that. Similarly, he has made up his mind on GNOME Shell it is “McCann’s” plaything, design suggestions made early in the process were ignored, and even though he now admits that the result is better than the early mock-ups, it is clear to me that there is no chance that Ubuntu will ever voluntarily adopt GNOME Shell.

To solve the second problem, we need to start using names instead of project names. It’s too easy to ascribe ill-will to an anonymous faceless project, it’s another thing to do so to an individual with a name & a face. GNOME didn’t reject the StatusNotifier spec – two GNOME contributors on the xdg@freedesktop.org mailing list who read the spec (and who were in a position to do something about it) felt that their concerns were getting short-changed, and disengaged. I’d wager that 90% of GNOME project members didn’t even know about xdg before this week. Let’s call each other by our names (and be nice while we’re at it).

Behind closed doors conversations are poison

Another major issue we’ve had is a distinct lack of tracability. It isn’t helped by many Canonical developers using infrastructure which the appropriate GNOME developers don’t – but in fact I don’t care what public forum you use to develop & talk about your software. What is harmful is what I’ve dubbed the Water Cooler – when too many conversations happen in private email, at conferences over drinks, or on IRC, there is no tracability. One example: A key event in the timeline of shell/notifiers/Unity was the UX hackfest in 2008. Different people say different things were said. And there is no written trace of agreements or proposals concerning notifications. I was told that there were some conversations after the conference on IRC, but nothing was sent to a mailing list or recorded in a wiki page.

A member of the GNOME Shell team recently, in response to some questions about design decisions, said that a lot of the discussions & reasons behind the design would have happened on IRC. So there is no trace.

I have personally been involved in dozens of OTR discussions with Canonical people. I recall one where Jono urged me to “step up to the plate” and provide technical leadership for GNOME – my response was that that wasn’t the role of the board, that the distributors who depended on GNOME for their products had to set the lead. It would have been nice to have the discussion in public.

I know that writing stuff down after the fact is a pain. But it is required to allow for a traceability of community decisions and agreements – and also to highlight misunderstandings. If I had a dollar for every time I’ve had a conversation with a client about a technical spec, and he understood something different than me, I’d have enough money for a nice meal. By writing down understandings in clear and unambiguous terms, you are also giving the other party to the discussion the opportunity to correct misunderstandings early.

I also understand that there is an interest in putting on a good face, and not airing your dirty laundry in public (ironic, eh?) – for the past few years, the party line in Canonical has been “We love GNOME, we’re a GNOME shop” while behind the scenes there have been heartfelt conversations about the various problems which exist in GNOME & how to address them. The problem is, because these discussioons happen behind the scenes, they stay there. We never get beyond discussions, agreeing there is a problem, but never working together on a solution.

The party line in GNOME & KDE has been “we’re all pulling in the same direction, we like each other” – and for the most part it’s true. But if this week has shown nothing else, it’s shown that senior members of the KDE & GNOME projects actively mistrust each others motives & don’t believe we have the same interests at heart. At least, this is clear from Aaron Seigo’s comments on his blog post.

First, design is not code. Design documents can behave like code, however, with peer review and an iterative process, and can be wedded to the process of developing code, evolving as technical constraints and schedule pressure get in the way of the original design. Good designers work with coders – and this is how it happens in Canonical too.

However, Canonical has occasionally opted to create new projects, housed in Launchpad, rather than engage existing projects to evolve them. libappindicator is an example – several people suggested that it should/could be part of GTK+ – what changed between January 2010 when Ted Gould said ” I’d like to think that the code in libappindicator would useful, and maybe even migrate into a replacement for GtkStatusIcon in GTK+” and February 2010 when he wrote ” Q: Shouldn’t this be in GTK+?
A: Apparently not”?

Canonical has a policy that Canonical development is done in Launchpad, using Bazaar. Sometimes that’s fine – if you’re originating a project, then you get to choose the infrastructure. Bazaar & Launchpad are working just fine for a plethora of projects. But when you are working with other projects, you need to be where they are.

Cody Russell, long-time GNOME contributor, former Canonical developer, and the developer of client side decorations for GTK+ among other things, wrote in a comment to Aaron’s blog:

CSD is really not a good example of how stuff development between Canonical and GNOME should work. I’m the person at Canonical who started CSD, but never finished it.

It started as just an experimental hack, and somehow got picked up as a “Canonical project”. Once that happened my immediate manager told me to stop committing code to GNOME git and do any further work on it privately in bzr.

For me this made developing it further much more difficult, because it was an extremely large and intrusive change into GTK+ source code and my manager didn’t want upstream developers to help me with at least peer code review.

Apparently there was originally some desire to have libappindicator developed as part of GTK+. I don’t know why this did not happen, but perhaps the quote above can give some insight into why the project was developed as an independent module.

Similarly, having a discussion on a freedesktop.org list does not ensure that you are getting appropriate cross-platform buy-in for your ideas. There is no guarantee that you are talking to the right people.

Most free software developers I know are on lots of mailing lists, and for all but a small number directly related to their day-to-day work, they glance over them. I certainly fall into this camp – there are about half a dozen lists I’m on where I will open maybe only 1 in 10 emails, with a subject that looks like it might concern me directly. If you want to directly get my attention outside of that, a personal email, IRC ping or IM asking me to comment on something is a good way to get my attention.

In the early days of freedesktop.org, this is how things worked. There were well defined problems that needed solving, and the people concerned made a conscious effort to get the right people into a central desktop-agnostic mailing list. As time goes on, maintainerships evolve, people change jobs, new people arrive – there is no longer any guarantee that the people on the freedesktop.org mailing lists are the best people to be talking to.

Moving on

So where do we go from here? Well, first GNOME 3. We have a release coming up, and so does Ubuntu, and we’re both going to have a bumpy ride for the next few months, so that is presumably going to be the priority for everyone.

After that, the Desktop Summit will be an opportunity to start building bridges. We’ve made an effort this year to avoid tribalism in the conference, by framing the call for papers according to problem area (multimedia, mobile, platform, etc.) rather than by desktop. We will be continuing this, I’m sure, through paper selection and drafting the agenda. That said, you can bring a horse to water, but you can’t make him drink.

Looking through the list of headings here, a number of them are easily fixable, a number of them are much more troubling, and a result of letting discontent fester for months or years.

We can certainly improve the operation of freedesktop.org – currently there is no freedesktop.org as such, it’s a wiki & a mailing list server. To improve it, there needs to be a process whereby things are agreed, and a way to ensure that all concerned parties are engaged in that process. There were discussions about this in Gran Canaria, including members of both the GNOME and KDE projects. But to evolve freedesktop.org, buy-in of a number of key GNOME developers is essential – I can’t imagine any long-term changes happening without Owen Taylor’s agreement, for example.

We can increase the transparency of the operation of individual GNOME modules. This is one of the things I hoped to achieve last year with the GNOME census. By identifying the key contributors for each module, and the processes under which each module operates, we can help reduce the friction when people try to figure out how to work with GNOME. Ideally, something like Jon’s guide to the kernel will help reduce the number of dropped or unreviewed patches, make it easier for people to see what kinds of contributions would actually be welcomed by module maintainer teams, and help people figure out how to gain influence in a specific module and eventually become a maintainer themselves.

We must reduce the amount of back-channel discussions between the various project participants. Any important decisions or agreements that happen off-line must get written down & agreed to after the fact. IRC usage has become predominant in some teams, resulting in a lack of transparency of operation – GNOME should adopt the Apache policy of “if it didn’t happen on the mailing list, it didn’t happen” and encourage companies who want to encourage change in the project to do the same. I would appreciate all participants committing to a general policy of releasing design specs and code early to peer review – and in the case of Canonical, working upstream before working in their own distribution.

I think there is a potential for a GNOME Design group, for example, with qualified designers in a publicly archived, but invitation-only, mailing list, to allow design collaboration without a high level of poor quality amateur participation which has typified public usability or design lists.

Finally, smaller, focused teams, started on a case by case basis, will serve us better than long-living “collaboration” mailing lists like desktop-architects or xdg. To ensure that the right people are at the table, they need to be invited, and their presence needs to be documented, on a project-by-project basis. Of course discussions on these lists should be publicly archived, but they should only be useful as long as the specific problem area is being addressed, and should die a natural death afterwards.

That’s the easy stuff.

The more difficult issue is that we have allowed relationships to degrade so far. It feels like GNOME & Canonical are in a bad relationship – we used to love each other, and now every time we talk it seems like we’re speaking a different language. For a while, it seemed like GNOME & KDE contributors were working productively together & overcoming some of the historical issues between the projects, but over the past 3 years, it’s been clear that the progress we had achieved was illusory, and that deep-seated ill-feeling among a small number of project leaders have ensured that any early progress has been squandered.

Solving this problem is much more difficult, if it’s solvable at all. Change inside the GNOME project can only come from the grass roots, and the same goes for KDE. Adopting a code of conduct is less important than actually being nice. Too many people confusing being rude and abrasive as being terse and efficient. And getting a critical mass of community leaders in the same place at the same time to concentrate on common issues and approaches to solving them is difficult when there is so much pent up frustration and ill-will involved.

The Desktop Summit will be an important meeting point this year, where hopefully some of these issues can be resolved. In the meantime, I hope that we can start some small conversations soon to get people talking and trusting each other’s motives again. It will be a long and arduous process, and will require everyone to accept part of the blame for the situation we find ourselves in, and to accept the better days are possible.

As I said to Jono Bacon yesterday when he suggested we should all just get along & stop digging up the past: “Those who ignore the past are doomed to repeat it. And we have been failing for some time to understand the issues which people have working in freedesktop.org, or with GNOME, KDE or Canonical”. Unlike some people commenting on the various blog posts this week, I think that getting some of the dirty laundry out into the open will be beneficial to the general working environment. Sunlight is a great disinfectant, they say, and a number of issues have been kept under wraps too long by people who want to put a brave face on & pretend for public benefit that everything is rosy.

Well, everything isn’t rosy, and now even a fool could see that. But I don’t think there’s anything broken that we can’t fix. Let’s concentrate on getting GNOME 3.0 and Ubuntu 11.04 out the door, and then get to work mending relationships.

For the app indicators we also had a lot of community involvement, it was based on a Freedesktop.org spec, worked on with consultancy from KDE, we invited GNOME developers to participate in the Freedesktop discussion and proposed them to the GNOME community for inclusion, but it’s not up to us, if they take it or not

Where tensions between Canonical and GNOME have occurred, according to Bacon, is in Canonical’s desktop innovations for improved usability, such as the Ayatana indicators for sound and social media, and the new Unity desktop, all of which were submitted to GNOME and rejected, leaving Canonical to develop them outside the GNOME project. […] Asked whether Canonical could have developed its usability modifications within GNOME, he replies, “To be honest with you, I don’t think it could have been done. The fact that nothing’s been accepted is a pretty reasonable indicator that the two projects have widely different directions.”

We committed to build Unity […] because we had ample reason to believe that the trajectory of the alternatives was going to fail. And it did fail – Gnome 3 looks much more like the vision we painted with Unity than the original vision […] I am sorry that a few Gnome leaders have blocked Gnome’s adoption of Unity API’s, and the stress that will cause, but I feel proud that we had the guts, and the capacity, to design and deliver something wonderful.

I have seen and heard this mentioned by others too, but cannot find any others right now – additional pointers in comments would be welcome!

So – given that GNOME is a project which scores very highly as being Open By Rule (disclosure: I put together the evaluation of GNOME for Simon), I thought I would go back through the archives and see how true this was.

Looking at what was actually proposed for inclusion in GNOME from Ayatana work, libappindicator was rejected because (quoting directly from the release team’s decision):

it doesn’t integrate with gnome-shell

probably depends on GtkApplication, and would need integration in GTK+ itself

we wished there was some constructive discussion around it, pushed by the libappindicator developers; but it didn’t happen

there’s nothing in GNOME needing it

I went back to see where the discussion happened for the libappindicator proposal. There was a discussion, some over & back, Ted was (as usual) forthcoming & helpful, and things appeared to be moving approximately in the right direction. There were some issues over copyright assignment, and the discussion petered out. No feedback I could see from the GNOME Shell team – positive or negative – to depending on the library.

Now, this is hardly ideal. I would love to see debate on why there wasn’t a more in-depth debate on using libappindicator in GNOME Shell. Was this ever proposed? If so, where? I can’t find any reference. Was there any reaction other than “we don’t think it’s an issue” to the copyright assignment issue? Perhaps there was a lot more discussion in another forum that I haven’t linked to – on the release-team list, on IRC, or elsewhere? Comments, please!

I would love to point to other instances of work which has been proposed upstream from Canonical and which has been rejected, but my (admittedly, brief) search has not turned up much useful stuff. I can’t find any online reference to displeasure with the GNOME Shell vision, or proposals of alternatives, nor can I find situations of “Paper Cut” patches being rejected because they were from Canonical or Ubuntu. In fact, the one reference I found to the UX hackfest in 2008 from Mark seemed quite positive about the whole thing.

There are apocryphal stories about patches submitted twice by different people before they were accepted, other stories about people being “impossible to work with”, design feedback being ignored, and more – I would love to see some evidence of this, or some documented criticism from 2008 of some of the GNOME Shell design documents. I hear often that some of the design decisions were unacceptable, but ask which ones, where the discussion took place, or how much effort was spent trying to get things changed, and hand-wavy “lots of stuff” type answers is what you get back.

I would really like to shed some sunlight on this – if we do not have publicly archived references to places where these disagreements have happened, then there are a couple of possible conclusions we can draw: either insufficient effort was made to collaborate, or the effort was made, and GNOME Shell is not sufficiently transparent for the developers and designers to be accountable.

So please – pile in on the comments. I want to know of instances when GNOME has (allegedly) refused contributions or help from Canonical, with links to Bugzilla, mailing lists, even IRC logs or wiki pages. Let’s get to the bottom of this & see if we can’t solve the problem.

Updated to clarify that the reasons for rejecting libappindicator were not mine, but were copied from the release team decisions, after reading Aaron Seigo’s response

The post-Elopocalypseangst has been getting me down over the past few days. It’s against my nature to spend a lot of time worrying about things that are decided, done, dusted. It was Democritus, I think, who said that only a fool worries about things over which he has no control, and I definitely identify with that. It seems that a significant number of people on mailing lists I’m subscribed to don’t share this character trait.

I prefer to roll with the punches, to ask, “where do we go from here?” – we have a new landscape, with Nokia potentially being a lot less involved in MeeGo over the coming months. Will they reduce their investment in 3rd party developers? Perhaps. I expect them to. Will they lay some people off? I bet that there will be a small layoff in MeeGo Devices, but I’d wager that there will be bigger cuts in external contracts. In any case, this is something over which I have no control.

First up – what next for MeeGo? While MeeGo is looking a lot less attractive for application developers now, I still think there’s a great value proposition for hardware vendors to get behind it in vertical markets. Intel seem committed, and MeeGo (even with Nokia reducing investment) is much broader than one company now. A lot of people are betting the bank on it being a viable platform. So I think it will be, and soon.

Will I continue contributing time & effort to MeeGo? My reasons for contributing to MeeGo were not dependent on Nokia’s involvement, so yes, but I will be carefully eyeing business opportunities as well. I’d be lying if I said that I didn’t expect to get some business from a vibrant MeeGo ecosystem, and now I will need to explore other avenues. But the idea of collaborating on a core platform and building a set of free software form-factor specific UIs is still appealing. And I really do like the Maemo/MeeGo community a lot.

Luckily, the time to market difficulties that Nokia experienced are, in my opinion, issues of execution rather than inherent problems in working with free software. Companies have a clear choice between embracing proprietary-style development and treating upstream as “free code” (as Google have with Android), or embracing community-style development and working “The Open Source Way” (as Red Hat have learned to do). Nokia’s problems came from the hybrid approach of engage-but-keep-something-back, which prevented them from leveraging community developers as co-developers, while at the same time imposing all the costs of growing and supporting a large community.

I expect lots of companies to try to learn from this experience and start working smarter with communities – and since that’s where I can help them, I’m not too worried about the medium term.

I would bet on Nokia partners and subcontractors battening down the hatches right now until the dust settles, and potentially looking for revenue sources outside the MeeGo world. If I had a team of people working for me that’s what I’d do. If some Nokia work kept coming my way, I’d be glad of it, but right now I’d be planning a life without Nokia in the medium term.

For any companies who have followed Nokia from Symbian to MeeGo, my advice would be to stick to Linux, convert to an Android strategy, and start building some Windows Phone skills in case Nokia’s bet works out, but don’t bet the bank on it. And working effectively with community developed software projects is a key skill for the next decade that you should be developing (a small plug for my services there).

For anyone working on MeeGo within Nokia, the suspense over who might lose their jobs is worse than the fall, let me reassure you. Having been through a re-org or two in my time, I know that the wait can last weeks or months, and even when the cuts come, there’s always an itching suspicion of another one around the corner. Nothing is worse for morale in a team than wondering who will still be there next month. But you have learned valuable and sought-after skills working on MeeGo, and they are bankable on the market right now. If I were working on MeeGo inside Nokia right now, I think I’d ignore the possibility of a lay-off and get on with trying to make the MeeGo phone as great as possible. If I got laid off, I’d be happy to have a redundancy package worthy of Finland, and would be confident in my ability to find a job as a Linux developer very quickly.

For community members wondering whether to stick with MeeGo or jump ship, I’d ask, why were you hanging out around MeeGo in the first place? Has anything in the past week changed your motivations? If you wanted to have a shiny free-software-powered Nokia phone, you should have one by the end of the year. If you wanted to hack on any of the components that make up MeeGo, you can still do that. If you were hoping to make money off apps, that’s probably not going to happen with MeeGo on handsets any time soon. If you’re not convinced by the market potential of MeeGo apps on tablets, I’d jump ship to Android quick (in fact, why aren’t you there already?).

Qt users and developers are probably worried too. I don’t think that Qt is immediately threatened. The biggest danger for Qt at this point would be Intel & others deciding that Qt was a bad choice and moving to something else. That would be a massive strategic blunder – on a par with abandoning the GTK+ work which had been done before moblin 2 to move to Qt. Rewriting user interfaces is hard and I don’t think that Intel are ready to run the market risk of dropping Qt – which means that they’re pot-committed at this point. If Nokia ever did decide to drop Qt, Intel would probably be in the market to buy it. Then again, I can also see how Qt’s management might try to do an LMBO and bring the company private again. Either way, there will be a demand for Qt, and Qt developers, for some time to come.

No-one likes the guy giving unwanted advice to everyone, so this seems like a good place to stop. My instinct when something like this happens is to take a step back, see what’s inherently changed, and try to see what the landscape looks like from different perspectives. From my perspective, the future is definitely more challenging than it was a week ago, but it’s not like the Elopocalypse wiped out my livelihood. In fact, I have been thinking about life without Nokia since MeeGo was first announced last year, when I guessed that Nokia would prefer working through the Linux Foundation for an independent eye.

But even if Nokia were my only client, and they were going away tomorrow, I think I could probably find other clients, or get a job, quickly enough. It’s important to put these things in perspective.

One of the most important documents a project can have is some kind of elaboration of what the maintainers want to see happen in the future. This is the concrete expression of the project vision – it allows people to adhere to the vision, and gives them the opportunity to contribute to its realisation. This is the document I’ll be calling a roadmap.

Sometimes the word “roadmap” is used to talk about other things, like branching strategies and release schedules. To me, a release schedule and a roadmap are related, but different documents. Releasing is about ensuring users get to use what you make. The roadmap is your guiding light, the beacon at the end of the road that lets you know what you’re making, and why.

Too many projects fall into the trap of having occasional roadmap planning processes, and then posting a mighty document which stays, unchanged, until the next time the planning process gets done. Roadmaps like these end up being historical documents – a shining example of how aspirations get lost along the way of product development.

Other projects are under-ambitious. Either there is no roadmap at all, in which case the business as usual of making software takes over – developers are interrupt-driven, fixing bugs, taking care of user requests, and never taking a step back to look at the bigger picture. Or your roadmap is something you use to track tasks which are already underway, a list of the features which developers are working on right now. It’s like walking in a forest at night with a head-light – you are always looking at your feet avoiding tree-roots, yet you have no idea where you’re going.

When we drew up the roadmap for the GIMP for versions 2.0 and 2.2 in 2003, we committed some of these mistakes. By observing some projects like Inkscape (which has a history of excellent roadmapping) and learning from our mistakes, I came up with a different method which we applied to the WengoPhone from OpenWengo in 2006, and which served us well (until the project became QuteCom, at least). Here are some of the techniques I learned, which I hope will be useful to others.

Time or features?

One question with roadmaps is whether hitting a date for release should be included as an objective. Even though I’ve said that release plans and roadmaps are different documents, I think it is important to set realistic target dates on way-points. Having a calendar in front of you allows you to keep people focussed on the path, and avoid falling into the trap of implementing one small feature that isn’t part of your release criteria. Pure time-based releases, with no features associated, don’t quite work either. The end result is often quite tepid, a product of the release process rather than any design by a core team.

I like Joel’s scheduling technique: “If you have a bunch of wood blocks, and you can’t fit them into a box, you have two choices: get a bigger box, or remove some blocks.” That is, you can mix a time-based and feature-based schedule. You plan features, giving each one a priority. You start at the top and work your way down the list. At the feature freeze date, you run a project review. If a feature is finished, or will be finished (at a sufficient quality level) in time for release, it’s in. If it won’t realistically be finished in time for the release date, it’s bumped. That way, you stick to your schedule (mostly), and there is a motivation to start working on the biggest wood blocks (the most important features) first.

A recent article on lessons learned over years of Bugzilla development by Max Kanat-Alexander made an interesting suggestion which makes a lot of sense to me – at the point you decide to feature freeze and bump features, it may be better to create a release branch for stabilisation work, and allow the trunk to continue in active development. The potential cost of this is a duplication of work merging unfinished features and bug fixes into both branches, the advantage is it allows someone to continue working on a bumped feature while the team as a whole works towards the stable release.

Near term, mid term, long term

The Inkscape roadmap from 2005 is a thing of beauty. The roadmap mixes beautifully long-term goals with short-term planning. Each release has a by-line, a set of one or two things which are the main focus of the release. Some releases are purely focussed on quality. Others include important features. The whole thing feels planned. There is a vision.

But as you come closer and closer to the current work, the plans get broken down, itemised further. The BHAGs of a release in 2 years gets turned into a list of sub-features when it’s one year away, and each of those features gets broken down further as a developer starts planning and working on it.

The fractal geometer in me identifies this as a scaling phenomenon – coding software is like zooming in to a coastline and measuring its length. The value you get when measuring with a 1km long ruler is not the same as with a 1m ruler. And as you get closer and closer to writing code, you also need to break down bigger tasks into smaller tasks, and smaller tasks into object design, then coding the actual objects and methods. Giving your roadmap this sense of scope allows you to look up and see in the distance every now and again.

Keep it accurate

A roadmap is a living document. The best reason to go into no detail at all for future releases beyond specifying a theme is that you have no idea yet how long things will take to do when you get there. If you load up the next version with features, you’re probably aiming for a long death-march in the project team.

The inaccurate roadmap is an object of ridicule, and a motivation killer. If it becomes clear that you’re not going to make a date, change the date (and all the other dates in consequence). That might also be a sign that the team has over-committed for the release, and an opportunity to bump some features.

Leave some empty seats

In community projects, new contributors often arrive who would like to work on features, but they don’t know where to start. There is an in-place core team who are claiming features for the next release left & right, and the new guy doesn’t know what to do. “Fix some bugs” or “do some documentation” are common answers for many projects including GNOME (with the gnome-love keyword in Bugzilla) and LibreOffice (with the easy hacks list). Indeed, these do allow you to get to know the project.

But, as has often been said, developers like to develop features, and sometimes it can be really hard what features are important to the core team. This is especially true with commercial software developers. The roadmap can help.

In any given release, you can include some high priority features – stuff that you would love to see happen – and explicitly marked as “Not taken by the core team”. It should be clear that patches of a sufficiently high standard implementing the feature would be gratefully accepted. This won’t automatically change a new developer into a coding ninja, nor will it prevent an ambitious hacker from biting off more than he can chew, but it will give experienced developers an easy way to prove themselves and earn their place in the core team, and it will also provide some great opportunities for mentoring programs like the Google Summer of Code.

The Subversion roadmap, recently updated by the core team, is another example of best practice in this area. In addition to a mixed features & time based release cycle, they maintain a roadmap which has key goals for a release, but also includes a separate list of high priority features.

The end result: Visibility

The end result of a good roadmap process is that your users know where they stand, more or less, at any given time. Your developers know where you want to take the project, and can see opportunities to contribute. Your core team knows what the release criteria for the next release are, and you have agreed together mid-term and long-term goals for the project that express your common vision. As maintainer, you have a powerful tool to explain your decisions and align your community around your ideas. A good roadmap is the fertile soil on which your developer community will grow.