open source marketer and community manager

development

Sometimes I like to sit down and play with technology. My colleague Mike Shroder mentioned VVV a few weeks ago and I had to try it. Check out this tutorial I wrote on how to create a new WordPress site, use Vagrant and git to develop a new theme locally, then push the modifications to a live site running on DreamHost (but it could run anywhere else, really).

I’m starting to feel the pain of developers attempting to write applications on top of OpenStack clouds: it hurts, and I haven’t come anywhere close to doing anything special.

I started following the getting started tutorial for First App Application For OpenStack (faafo) and I gut stuck on step 1, authentication. There are way too many things that are convoluted, confusing and sometimes carefully hidden (or ridiculously displayed) on Horizon that make it too hard to get started developing apps on OpenStack. I started following the Get Started document for python-libcloud:

You need the following information that you can obtain from your cloud provider:

auth URL
user name
password
project ID or name (projects are also known as tenants)
cloud region

Looks simple but it’s not. The auth URL doesn’t include the path say libcloud documentation but in practice every cloud I tried behaves differently. DreamHost and Runabove require not only v2.0/ but also /tokens after the base URL (https://keystone.dream.io/v2.0/tokens) while CityCloud and HPCloud seem to work according to the documentation. Some will throw weird errors if you add /tokens. Libcloud is also confusing regarding support for Keystone API v3 (which Citycloud uses): the official docs don’t mention v3 (bug), but a blog post from libcloud maintainer provides examples on how to use it.

Finding the right projectID or name is also challenging because some clouds don’t show the project ID immediately in the web control panel (and not every cloud I tested runs OpenStack Horizon). Chances are you’ll have to find the RC file, which is fairly easy if the cloud you’re targeting is using Horizon.

Even after I found all the pieces, I haven’t managed to authenticate using libcloud on CityCloud, Rackspace (using openstack provider) and HP. Only with OVH I managed to authenticate and get a list of images with my stock Ubuntu 15.04. I managed to authenticate on DreamHost only on Ubuntu 14.04 and on a clean, updated virtualenv.

It took a lot of trials and errors to get rid of the Method Not Allowed errors and get libcloud.common.types.InvalidCredsError: ‘Invalid credentials with the provider’, which at least is hinting that I have guessed the auth_url for Citycloud and HP. But I have no idea why credentials are not accepted, since the username and password are those I use on the Horizon panel on HP and on on the custom Citycloud panel. My guess is that I haven’t guessed correctly the project_name or tenant_id or whatever it’s called. This is too confusing.

it won’t work without /v2.0/tokens. So, after spending one day reading docs, trying to guess permutations I started thinking that probably libcloud is not a feasible approach.

I’ve started looking at OpenStack Infra homegrown interoperability library shade instead: authentication went smoothly immediately with DreamHost and HP on my machine. At least that part was easy and I managed to make some progress in my test with that, finally. Hopefully by the end of the day I’ll have the FAAFO running on

The end result is that it shouldn’t be this hard, even for a very modest developer like me, following a well written, copy-and-paste tutorial should not be as confusing. A lot of work needs to be done to make OpenStack clouds friendly to app developers.

I’m cruising Silicon Valley these days asking “how does your company do OpenStack?” to collect best practices and notable mistakes from various leaders of OpenStack corporate community. I’m hoping to build a ‘how to’ manual to help managers build better dev teams, more effective at collaborating while shipping products to their customers. This is an effort that goes hand-in-hand with training new developers with Upstream Training and other initiatives aimed at sustaining OpenStack growth.

I often have to explain to managers of OpenStack corporate members how and why to contribute to the project. The level of complexity reached has now exceeded the simple answers, like ‘fix simple bugs’ or ‘read the wiki pages’. For example, the participants to the Upstream Training in Atlanta had difficulties finding low-hanging-fruit bugs to work on: by the time they were ready to commit, someone else had already fixed it. Too much is going on at the same time, newcomers are confused at many different levels.

While we will keep training developers through Upstream Training (and other initiatives), we’re going to start talking to managers too. I’d like to know how development teams are organized, how they balance shipping products to customers with contributing upstream, what incentives are in place that help or prevent contributions to go upstream, how career paths are influenced by contributions to the open source collaboration, etc.

No doubt that OpenStack is one of the largest open source development efforts existing at the moment, with Linux, Debian and few others. What can be improved to make the OpenStack community grow faster? How can we help new contributors become productive more rapidly?

Whatever dimension you look at, the growth of the OpenStack developers community is astonishing. I’ve looked at the reasons for this growth and I’ve identified four main causes summarized in my presentation at Linux Conf North America and Open World Forum in Paris last year:

One clear mission to solve a hard problem

One initial core team with strong motivation to solve that problem

One team dedicated to recruiting new team members

One clear path to onboard new members in the team

The first two points are pretty clear already and don’t seem to need much tweaking: those ain’t broken, so no need to fix’em. The other two are not broken either but they’re the kind of stuff you want to stay on top of because once you notice they’re broken, the cost to fix them will be higher. Recently I started to ask myself what margins of improvements there are in recruiting and onboarding new members in our communities.

I quickly realized that while we know a lot about the top contributors to OpenStack, we don’t know much about the ‘long tail’ of contributors. It’s fairly easy to understand why the people responsible for 80% of OpenStack code and documentation work on the project. But who are the occasional contributors and what are their motivations to join the community? I embarked on a quest to get to know this long tail in an effort to understand how we can improve recruiting and onboarding new developers.

After a conversation with Asheesh Laroia, of OpenHatch fame, I selected 55 random developers from the list of those who submitted a single change to the integrated OpenStack programs during the past 12 months. These people went through quite a huge chunk of trouble only to submit one single patch: besides learning enough of the quite complex OpenStack code, they had to sign the Individual CLA (maybe the Corporate CLA, too), learn about git review and the other tools, go through the code review process. I would expect that whoever goes through that process, does it with the intention to stick around. So I asked the sample two simple questions:

What made you decide to submit a patch to OpenStack?

What prevented you to submit more patches?

I’ll wait a few days for their answers and share more details. Feel free to answer the questions leaving a comment here, too.

I’ve discussed recently with a friend of mine photographer, illustrator and animator about the status of GIMP, Inkscape and Blender. The good news is that professionals increasingly know about these free software tools, which is already a great step forward compared to the past years. Pierpaolo acknowledged how powerful all of them are but also noticed how different they are all from other similar software in the same field. It occurred to me that while other desktop tools like Open/LibreOffice have ways to raise money to finance the development of new features, improve the user experience and interface, etc Gimp and Inkscape are primarily developed by volunteers (Blender’s development is financed by the non profit Blender Foundation through grants and donations). This whole led me to think again about how hard it is for free software projects to invest time and energy in refactoring the GUI when there are so many cooler things to add to the core functions of the software (think of the eternal complaint about quadricromy support in GIMP). Would these be interested in improving their UI if they had more money available or if they had actual ‘customers’ instead of users?

When I was thinking about all this I learned that Sourceforge released a new program to fund development of free/open source software with a revenue sharing program called DevShare. Reading the press release, DevShare offers free software developers the option to bundle extra software with their downloads and share revenues with SourceForge. When a user downloads FileZilla for example, she’s offered the option to install also another piece of software with FileZilla. SourceForge is not the first site to offer bundled downloads but it does it with a better approach, avoiding traps. They looked at best practice policies to avoid confusing end-users with misleading installation flows and promises to provide clear documentation and procedures to uninstall undesired applications.

The revenue sharing with the developers is what is most interesting to me: developers who voluntarily decided to join similar programs are often required to spend time integrating their applications with third party installers, and have limited control over what and how that’s offered to their end-users. SourceForge’s program on the other hand seems to be very open and transparent towards the developers. I’ll be following the evolution of the program, hoping that free lance open source developers find motivation.

Back from Italian Agile Day where Stefano Fornari of Funambol with Marco Abis of Sourcesense animated a debate about mixing Free/Libre Open Source Software (FLOSS) and Agile development methods. I used to think that there was no issue because, after all, free software is a way to release software and it’s not a development method like many still think. Strictly speaking, what makes software free and open source is its license, not how it’s developed. But a lot of FLOSS is indeed developed in similar ways, with distributed teams, volunteer based contributions, merithocracy based leadership and so on. Some of these traits make FLOSS and Agile difficult to mix.

At Funambol we love Agile, me included, and we love to try new things so we proposed an experiment mixing Agile methods with community based development into a new Funambol Code Sniper program. The slideshow below summarizes the basis of this experiment based on the assumption that the community is the Product Owner of the new software. The community will have to define the user stories and also to define when they’re DONE.

There are still a few grey areas, the biggest being how to distribute rewarding to contributors. I think they should be proportionate to the efforts put into the project. Even if it is possible to evaluate code contributions proportionally to story points (or hours/weeks), code is only a part of software development. Bug reporting, quality assurance, feedback and even writing user stories is important as well: how to evaluate these other kind of contributions? What do you think?

Gianugo Rabellino has given me more food for thoughts about my research on Free/Libre Open Source software development and Agile/Scrum methods. His latest post contains a sentence that summarizes my key finding so far:

At the end of the day, this means that the customer is there – it just happens to coincide with the community as a whole.

Talking with my Funambol colleagues, the pragmatic agilists, and looking at Ross Gardler presentation below, I have the confirmation that the Pentaho guys are on the right track with Open Scrum. I also learned that it’s better not to use the word Scrum if it’s not The Scrum you’re talking about. With that in mind, I’m now focusing on best practices for communication between developers distributed around the world (more in latest posts).

Funambol engineering team uses the SCRUM methodology to develop software. It’s a very interesting method that seems highly compatible with free/libre open source software development habits. It mandates fast release cycles (like the release early/release often mantra), teams that can self-organize. SCRUM also mandates fixed time (2 to 6 weeks) to complete a development cycle (called iteration or sprint). This last part doesn’t seem to be very compatible with contributions by volunteers.

I’ve been looking for other free software projects that use SCRUM internally to understand how they involve external contributors, volunteers, in strictly time constrained release cycles. Pentaho wiki has a very interesting paper on the topic, but I still don’t understand if they have established a process to assign user stories to volunteer contributors.

I wonder if some have tried and failed or nobody has ever tried this at all.

Nokia has shown the new UI framework for the next Maemo SDK, codename Freemantle. They finally are getting rid of the stylus keyboard and a lot of very cool new features. Ars published an overview and more up to date details. I’m a fan of the N8xx devices but I’m still waiting for devices with phone capabilities. Speaking of which, I think it’s time for a Funambol client to also run on Maemo. There is already Syncevolution, a powerful syncml client compatible with Funambol, but it’s missing a GUI. Funambol can offer $750 to develop a full graphic user interface for Syncevolution to run on the stable Maemo 4 or on the new Maemo 5. Do you know somebody interested in developing on Maemo? Go to the Funambol Code Sniper community for the details.