I had to wait after each git push to discover if my .gitlab-ci.yml file was valid or not.

As usual, automation is the answer. Wouldn't it be awesome if we could run this:

gitlab-ci-validate-watch

And then edit our .gitlab-ci.yml until it's valid? That would be really awesome right?

Sadly at the time of writing Gitlab-CI documentation and API requires us to send our YAML file in stringified JSON format inside a content object to /api/v4/ci/lint api endpoint. Yep, that's a lot of hard work for such a simple task.

Fortunately we can leverage jq.node (it's like jq but WAY MORE powerful) for that along with watchexec!

If you do not have jq.node and watchexec installed it's never too late:

npm i jq.node -g
brew install watchexec

With these two I was able to write the following helpers (don't forget to add them inside your ~/.zshrc or equivalent):

gitlab-ci-validate validates .gitlab-ci.yml file from the current working directory using gitlab.com (it will also work with self-hosted gitlab instances) and gitlab-ci-validate-watch runs gitlab-ci-validate every time I save .gitlab-ci.yaml.

gitlab-ci-validate
{
"status": "invalid",
"errors": [
"jobs:update:db:script config should be a string or an array of strings"
]
}

For extra sweetness, we might want to run gitlab-ci-validate before each git push using git pre-push hook.

⚡️ Receive a weekly recap' of the best libraries and projects I found on the web

Note: I definitely prefer to restart docker-compose up after each file change (with soft shutdown) than have to first remember to run docker-compose up and then run watchexec ... "docker-compose restart" and finally ctrl+c + docker-compose down.

⚡️ Receive a weekly recap' of the best libraries and projects I found on the web

What a long way since the first issue of RedisWeekly the 16th of August 2013!

As you may have seen from the past weeks, I was not able to send you weekly doses of Redis news! In this regard I decided to leave RedisWeekly in the good hands of Redis Watch curator Itamar Haber from Redis Labs and I would highly recommend you to stick with Redis Watch.

If you do not unsubscribe from this mailing list before the 9th of April you will be automatically subscribed to Redis Watch.

As for me I will continue to share and spread what I find interesting on my Twitter account and hope I will meet some of you at some tech conference!

Thank you for all these years of your trust in RedisWeekly.

⚡️ Receive a weekly recap' of the best libraries and projects I found on the web

Recent events aside, Gitlab and Gitlab-CI is a great integrated forge for software development. I recently decided to migrate Image-Charts on it as well as the soon-to-be-announced-new-SaaS from the old Bitbucket, Jenkins workflow used at Redsmin and Bringr.

As a side note, JenkinsFile never grew up on me, I never liked it, it's too verbose and I definitely prefer the configuration approach (limited feature-set) over the code (unlimited but can quickly get messy) approach.

I first tried to setup a private deploy SSH key as an environment variable and then inject it using GIT_SSH_COMMAND and then hack the known_hosts file to fix the sadly well-known Host key verification failed issue aaaand don't forget to chmod 400! Phew, that's a lot of work for something that should be easy to do. Thankfully there is a simpler way!

You will first need to install clever-tools locally (if you did not already). We could do the following steps without it but doing the oAuth dance through the API is not as easy as using Clever CLI.

deploy:clevercloud: the job name, could be deploy or whatever you want

image: node:6-wheezy: I used this docker image on the previous steps because the app is in NodeJS you can use any docker image you want as long as it has git installed

stage: deploy: gitlab-ci pipeline stage.

only: - /master/: restrict this job to the master branch.

git remote add clever ...: we first need to add CleverCloud remote git repository to the build local git repository.

... https://$CLEVER_TOKEN:[email protected]oud.com/...: this is where the magic happens, instead of using git+ssh we are using https transport, the authentication is through basic auth token:secret and thus we don't need to setup a private ssh key.

... git push --force clever master ... I always use --force in CD pipelines, I don't want anyone else to bypass the CD pipeline. It's often a source of longer outage when tests are bypassed to fix directly the production environment.

... 2>&1 | grep -e 'remote:' -e '->' ... this part is very important, without it token:secret will leak into job logs and even emails in case of job failure.

That's it! It only took 2 lines in a Gitlab-CI job to automatically deploy your project on CleverCloud.

Deploying to CleverCloud is only one side of the story, I hope to share later the Gitlab-CI pipeline I use to deploy the soon-to-be-announced-new-SaaS with Kubernetes on Google Container Engine.

⚡️ Receive a weekly recap' of the best libraries and projects I found on the web