The fact that Shippable runs inside of Docker means that it keeps a persistent state and every build will not have to revert to initial state where it needs to install every dependency from the ground up. Classic CI tools that run on virtual machines need to reset their environment every time and e...

Currently, Shippable does not allow for build artifacts to be natively deployed to S3. This can be gotten around, however it is a rather large hole when compared to Travis. In order to deploy to S3 you have to add a couple of lines to the yml file. For example: env: global:#secure variable...

Shippable runs inside Docker containers. Docker has some specific security measures which may or may not become a hindrance in using Shippable. It may be harder for users who are not very comfortable with a Linux container environment and that can create some security problems. Even for more advanc...

AppVeyor's configuration (which is done from the .yaml file in the root of the project) is unfortunately very limited. The configuration is either tied to a branch or, in other cases, it's global. This limits the developer to a single build process. However, since you can use arbitrary scr...

When registering with Bitbucket Codeship it requests way to many permissions, even "Read and write to your team's projects and move repositories between them". Before giving all these permissions you have to be sure you can trust this service.

Semaphore is not free and nor is it open source. Pricing starts at $29 per month. However, there is a free option for private projects which have less than 100 builds per month and it's free for open source projects.

Bamboo is the only build server to offer first-class support for the "delivery" aspect of continuous delivery. Deployment projects automate the tedium right out of releasing into each environment, while letting you control the flow with per-environment permissions.

When connecting Bamboo with Stash and JIRA, details like JIRA issues, commits, reviews and approvals follow each release from development to production. If HipCHat is part of the integration, team members get notified right away in addition to email notifications.

Bamboo allows using Docker containers to create build agents. Using Docker agents lets you run multiple remote agents on the same host without conflicting requirements. It makes it easier to duplicate and distribute changes to build agents, and to use scripts for creating and maintaining agents....

The tests of GitLab CI run parallel to each other and are distributed on different machines. Developers can add as many machines as they want or need, making GitLab CI highly scalable to the development team's needs.

Gitlab CI can only be integrated with Gitlab out of the box. In order to use it with another service (like GitHub or Bitbucket) with which you are hosting your repository, you have to use a third party service to make this possible. This adds additional cost and maintenence overhead.

CircleCI excels with its setup process. All that's needed is a GitHub login and CircleCI automatically detects the settings for Ruby, Python, Node.js, Java and Clojure. The setup process is their most widely praised feature.

CircleCI can be connected to any project that is hosted on GitHub by logging in using the GitHub OAuth and adding the desired repository. Whenever a new commit is pushed to GitHub, CircleCI runs the tests that have been already defined and if none of them fails, the build is deployed to the runti...

Currently (October 5th 2016), Docker installed on the VM is: 1.9.1-circleci-cp-workaround, build 517b158, and docker-compose is 1.5.2, build 7240ff3. docker-compose in particular is almost too old to be used.

Even though Jenkins is pretty functional and useful out of the box, there's a large plugin ecosystem from which the user can choose plugins to integrate into their Jenkins build. This is needed for when the user wants to extend any of the tool's features.

Unlike some of the simple and hosted alternatives, users need to host and setup Jenkins by themselves. This results in both a high initial setup time, as well as time sunk into maintenance over a project's duration.

There have been several complaints by users regarding the quality of the plug-ins found in Jenkins' official plugin repo. A lot of plugins found in the default plugin directory are no longer actively maintained and as a result, they may be incompatible with later versions of Jenkins or other pl...

Jenkins without plugins is almost useless. All plugins are treated equal and published almost right away. Because there is no process for testing Jenkins' integration, the overall Jenkins experience is not that great. Furthermore, Jenkins' core and plugins are released on a regular basis,...

There's no configuration files to write, nothing to download or install, nothing to really configure. Setting it up takes basically only adding your Git repository and waiting for the clone, build and test to finish.

Resources are to Concourse as plugins are to Jenkins. In other words, resources allow Concourse CI to do just about any work necessary in a build. But resources follow a "service provider interface" that makes them easy to build in any language (not just JVM languages) and have a clearly...

BOSH is an open source tool for release engineering, deployment, lifecycle management, and monitoring of distributed systems. Since Concourse CI is built on top of BOSH, Concourse can scale across many servers or be run in the Cloud.

The downside of building on BOSH is that a full, scalable deployment of Concourse CI requires AWS, vSphere, or OpenStack. If you don't already have these, any one of them can be a big effort to set up, just to get a build server running. Might not be a good fit for smaller teams.

Deployments are configured to run in sequence, one after one in the build pipeline. Not every environment is updated on every change, this is especially helpful for example for someone who is testing the application and does not want their environment to change while they are doing it.