We use the free version of Gitlab to host our Git repositories and launch the jobs that run all our testing. However, since we run the free version (and as a startup, we're cautious about our running costs), we are limited to what kind of CI we can run on Gitlab.

As such, we've installed and run our own Gitlab Runner, that uses Docker containers to run our testing. The Gitlab.com servers essentially instruct our servers to run the entire pipeline and we report back the status to Gitlab for reporting.

Setting up the local Gitlab Runner with Docker is pretty straight-forward and their documentation handles this perfectly. Some minor things we changed on our end to speed things up are related to the concurrency of the jobs.

Remember that multiple containers will run at the same time, potentially pushing your server to 100% CPU usage. For this reason we decided to run these on one of our test machines, even though production servers have a healthy abundance of free CPU and memory.

But, in order for us to handle spike usage, we need to keep those resources free & available.

If you have a stage that consists of 3 jobs (like our testing stage), remember that the 3rd job might take longer to complete with a concurrency of only 2 jobs. Those first 2 will run in parallel, the 3rd one will have to wait for a free job slot.

During our unit tests, we spawn a webserver to test several of our key features;

Uptime & downtime detection

We crawl that test server to detect mixed content & broken links

We test several of our custom HTTP header features

This test webserver spawns our public website, which in turn relies on compiled JavaScript & CSS (Laravel Mix). That gets compiled with webpack, which is why we run that asset-building stage before our unit tests.

Without it, our tests would simply fail as we can't render our homepage.

Crawling our own site - and various custom endpoints to simulate downtime or slow responses - has the additional benefit that we validate (most of) our website is functioning correctly before we deploy.

Some of our stages depend on the output of the previous stage in order to work. A good example is our testing stage. When we run phpunit, it will fetch oru homepage which in turn relies on the CSS & JavaScript that got generated in the previous stage.

Gitlab CI allows you to set your dependencies per stage.

phpunit:
[...]
dependencies:
- build-assets
- composer
- db-seeding

Setting the dependency makes sure the artifacts get downloaded from that particular job into this one, essentially copying the output of one job to another.

A cache is, as the word implies, a local cache of the data. It's available only locally on the server and is not guaranteed to be present.

An artifact is - in our own words - what you want to store in a job to pass on to the next one. Initially, artifacts were what you wanted to survive out of a job. Like generated logs or error files, PHP Unit coverage results etc.

But this feature can be used broader than just exporting test results: you can use it to pass the output of one job onto the next one.

We first thought to use caches for this purpose, as these get stored locally and are available faster. However, the cache isn't guaranteed to be there and about 30% of our pipelines would randomly fail because it was missing a composer vendor directory, compiled assets, ... even though those jobs completed just fine.

An obvious step missing here is - perhaps - the most important one: deploying the application.

We haven't implemented this in Gitlab CI yet. Our deployment is automated of course, but right now it's not tied to the status of our CI pipeline. We can deploy, even if the tests fail.

We're still a small team and we make the decision to deploy thoughfully, but controlled. As many Laravel users would use Envoyer for their deployment, further automation could be done to integrate that too.

We'll highlight our deployment strategy (and the reasoning to not couple this in with Gitlab CI) in a later blogpost, there are a lot of nuances that deserve highlighting for that.