Mobile CI tools review

I wrote this article originally for Codemate internal purposes, to evaluate different mobile CI tool options. It’s just a short summary of my opinions on the tools. I hope you find it useful.

There are plenty of mobile Continuous Integration services that offer pretty much the same set of ‘base’ options.

Basic features of the mobile CI services

Unit tests

Merge(pull) request tests

UI tests

Distribution

Meaning that basic needs will be completely covered when choosing any of the mobile CI tools reviewed in this article; buddybuild, Bitrise and Nevercode (formerly known as Greenhouse). We also evaluated Gitlab CI from a mobile perspective although it is not a mobile oriented CI system to start with.

Buddybuild is a continuous integration, continuous deployment, and user feedback platform for iOS and Android development.

It has pretty exclusive features for a CI platform, as it can integrate its own SDK on the build stage (no code modification needed) to give access to two additional components.

Firstly,Buddybuildgives the end user of the app the ability to send feedback to the developer team, starting with simply taking a screenshot of the app, writing a description, and/or highlight a part of the screenshot and send it to the developer (to buddybuild dashboard). Secondly, developers get a regular crash reporting feature.

Although aforementioned features are not exclusive to this service, they are rarely found in CI services.

On the other hand, sometimes it feels like it is lacking extensibility in areas of test deployment (email only) and triggers (basically it is build per push and build per push request – on/off). Which basically means that if one would want to set up a routine to distribute a new build for testers once in a while (not with every push, buddybuild!), one would need a separate branch for that, as build triggers can be configured on branch basis only.

Bitrise workflows deserve to be mentioned separately. Workflows allow seemingly to do anything what a developer would want and if there is no suitable workflow step defined one can create it by oneself.

Another handy thing is that Bitrise config is customisable not only through UI, but all workflow steps are stored in bitrise.yml file which can be modified manually if needed.

Yet another great feature is the CLI tool which allows us to execute workflows locally.

Pros and features

Extensibility

Create custom steps/scripts for a workflow

Plenty of pre-defined workflow steps

CLI to run workflows on local machine

Can execute Fastlane lanes

Command line tool to help iOS signing

Cons

No release notes

Nevercode (formerly known as Greenhouse) looks promising. The fastest build times of them all, simple and straightforward setup. However, at the time we tested it, it was still a pretty raw product with minimum features available.

Especially it lacks customisation on build triggering and deployment in particular, currently there is number of options to distribute the app, but no way to define a trigger for it, meaning if distribution is enabled it will be triggered with every successful build.

Pros

Fast

Simple

Autogenerated release notes

Neighbours 🙂

Cons

Too simple

Poor choice of build triggers

Gitlab.com, it is definitely not mobile oriented CI service, but it supports them. By using so called ‘runners’ it can build any Android or iOS project. Public projects get free access to all features and unlimited CI/CD, but on private projects the free plan is limited to 2,000 CI pipeline minutes per month on shared runners.

What Gitlab lacks completely is the distribution part, meaning it has basically no built-in means of mobile app distribution, except opening a build page manually and downloading zipped build artefacts.

A solution to this problem could be to setup Gitlab CI+Fastlane+distribution service, using some service solely for beta builds distribution. The downside is the increased complexity of project set up (requires configuration of Gitlab CI, Fastlane, as well as some third party service).

Pros

Free

Easy-to-control build environment

Next business day support with paid subscription

Familiar from web projects

Cons

No distribution

Non-trivial setup

Private projects may require dedicated runners

+/- Completely managed by us

Summary

Each of the reviewed mobile CI cloud services does its job diligently. Typical project set up is a breeze, as project type and its dependencies are detected and configured automatically.

Unit/UI tests and deployment is also supported. The biggest difference is the set of extra features on top of the basic ones. The tools being somewhat similar, it makes the final choice of the best mobile CI tool difficult and somehow biased, but for sure the finest set of options can be found in Buddybuild and Bitrise.

Costs

Gitlab also offers paid features for support.

The pricing may have changed since this post was made.
If you would like to report up-to-date information regarding these tools, or would recommend having other tools reviewed, please contact janne.salmi@codemate.com.

CODEMATE Bangkok

Follow Us

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible and to analyse our site and marketing performance. If you continue to use this website without changing the cookie settings on your browser you are consenting to this.