Information technology, applied.

Why I Do Not Like Jenkins

I recently installed Jenkins to use as a continuous deployment server for Ansible and serverspec. I did finally get it to work but what an astonishingly unpleasant experience it was. It was exactly the kind of experience that gives open source a bad reputation. I will post later how to make it work with Ansible and serverspec and then I will give Bamboo a try instead.

Installing it to CentOS from their repository was easy and I’ll give Jenkins’ developers credit for that. Sadly that was the last easy step in my Jenkins experience. As an Ansible user the first thing I look for is a way to automate the installation and configuration of the application. Well, get ready because automating this thing is going to take some serious time. Hope you don’t have a life. Just look at the complexity of this playbook. It only scratches the surface.

You can find its remote API docs here if case you have time to go spelunking. When the primary documentation is a wiki you know you are in trouble. Having users write your documentation is like letting interns run your company.

Jenkins is designed around its web interface. Like many other projects, its developers don’t realize that web based administration is OBE. Web based administration is holdover from the era of manual changes. Modern administrators need easy ways to automate the installation, configuration, and management of their applications. Probably the best is the simple text configuration file, a command line tool like nsupdate for BIND (or simple REST API) to manage running instances, and a way to monitor it remotely via a central console.

The real problem is that the web UI is terrible. Jenkins has adopted the Principle of Greatest Surprise for its UI. It never does what it says. You get to puzzle and rage over why. This requires endless and exhausting Googling. Normally UI creators try to avoid this. Here are some of the surprises I encountered.

The first thing I wanted to do is have Jenkins check out (‘clone’ in the opaque jargon of git) my source code. So I created a job and entered the git repository URL. I use SSH keys for git security so I then entered the private key in credentials which is in the field just below the URL. Guess what? That doesn’t work. Turns out out you have to manage SSH keys at the Linux user level. Uh oh, this could be a sign of things to come. And indeed it was.

Next I wanted to check out two repositories, one for my Ansible playbook and one for my serverspec tests. In the job config screen under git I saw that I coud add additional repositories. Cool, right? Wrong! That doesn’t work either. Why is that even there? Jenkins loves to do this: tease you with an obvious step and then make it not work. I am now starting to think this is an intentional design decision. It’s like some geek rite of passage, trial-by-crap-documentation. It turns out I have to download a plug-in called “Multipl SCMs” in order for this to work. Ok, Jenkins, strike two.

Finally, I want git to trigger a build when I ‘push’ a change to my git server. On the git side this is very easy. I just add a shell script with a wget call. On the Jenkins side, however, its business as usual: pain. On the job config page there is a checkbox to enable “Trigger builds remotely” by entering a token in a URL. So I add this token in my wget URL that git calls. Does it work? Of course not. I’m not even upset anymore. I have passed though all the stages of grief and am at the acceptance phase. Maybe I need to include my Jenkins username and password in the URL? That’s kind of a security breach but OK, I’ll try. Nope, no joy. After googling I learn of a plugin called “Build Token Root” that can make this work. This plugin, like “Multipl SCMs” is member of the work-around family of plugins designed to repair Jenkins’ basic behavior.

Unfortunately, it has poor documentation and I can’t get it to work either. How do you set the token? That’s it, I surrender. You win Jenkins. You are tougher than me. I want to spend my time with well-designed systems that make me productive quickly. Ansible is like that. serverspec is like that too. When I invest a lot of time in a product it is for the purpose of mastering complex and unusual features not for getting the basics to work.

Sounds to me like you didn’t spend enough time with Jenkins. Its interface is indeed horrible but you have ways to change that using Jenkins DSL plugin. It allows you to write projects in Groovy language and use variables, inheritance and so on which will make life easier. It does have some learning curve and IMO should be integrated but nothing that is show stopper.
Using Jenkins DSL you can keep all projects in repository and have 1 groovy file that will generate all Jenkins projects for specific deployment. Then, on blank Jenkins installation you can use REST API to create project which itself generate all other projects. That can be done via short script of several lines of code.

Having GUI interface is not a problem because most of the time Jenkins folks are sysadmins who have variable levels of code understanding. In Windows oriented company that almost the rule rather then exception so Jenkins GUI is far more friendly. So ideally, we do want both interfaces.

That is an interesting solution. I like the idea of using the Jenkins API, which looks like it would mean automating Jenkins just like the rest of the dev infrastructure. Can you comment of how difficult it was to implement?