Main menu

Category Archives: CI

Post navigation

Introduction

During the last article in this series on the use of PowerShell with GitLab, we noticed some displays on the build console, and some results from our build that was a bit unusual.

Today, I’m going to look a bit further into how the build engine works, which will also explain some of these results we are seeing.

How GitLab CI Runner Works

It would be logical to think that the PowerShell code is run in the same location or a subdirectory of the folder where the service file is, similar to Jenkins, but after a bit of investigation, I found out this is not the case.

To look further into what was happening, let’s add a line to the script section of the YAML file, and put a Start-Sleep command in.Make it ten minutes, to give enough time to remote onto the GitLab CI Runner, and do a search for our .ps1 file stored in a subdirectory of the repository.

Use the content below for our build file.

YAML

1

2

3

4

5

6

7

8

9

10

11

ps_helloworld:

script:

-functionHappyHelloWorld{

-$x="Hello world"

-Write-Output"I would just like to say $x"

}

-HappyHelloWorld|Out-File'c:\windows\temp\helloworld.txt'

-Start-Sleep-Seconds600

tags:

-windows

Save the changes

Add the updated file

Commit the changes

Push the changes

Navigate to the build section

Click on the commit

The job is running, and because of the Start-Sleep command, will continue to do so for the next ten minutes.

Discoveries

Remote onto your build server (if it is not already the same system you are accessing GitLab from.

Go to the C:\Windows\Temp

There will be a folder name of the format Build_xxxx

Double click on the folder

Within this folder is a script.ps1 file.

Open the file

The PowerShell code contained within the script: section of the YAML file is adjusted to make it suitable for returning the output we can see on the console window within a Gitlab job.

Looking at the code now, we can see that it has been changed significantly :

ErrorActionPreference has been set to ‘Stop‘.

Commands are in place to check to see if an error has occured, and if so to exit the script. This will also result in the job being flagged as having failed.

Probably the most interesting of the changes is the frequent use of Echo command which mimics the command that follows it.

This explains several things to us :

How the console output is seen on the job screen

Some lines output appeared strange because variables were undefined at the stage when the Echo command ran

Because PowerShell returns all output from a function, it explains why were receive an array when the HappyHelloWorld function is called. It consists of the intented output, but also the Echo commands.

The use of a function in our build code now becomes a challenge or possibly could be perceived as something to be avoided, since we cannot guarantee the element in the array that will contain our desired output. One way i’ve been able to get round this is to make the function return a PScustomobject, and make use of the -is operator outside of the function to obtain the value.

However, I’ve found a better approach is to minimize the actual PowerShell commands that you use in the script, and instead create a Build folder within your repository which contains files with the actual commands that you wish to use for the build. These can be dotsourced to make them available from within the script, without the need for adding custom code to handle this.

There are many other aspects and configuration settings possible within GitLab, which lets you have powerful control over your build steps. Examples of this are defining the order of execution, ignoring errors, and setting the steps to be taken after each one.

It’s also worthwhile noting that these articles do not include the use of testing, which should be part of your chain of operations. If you are not already familiar with the use of Pester, I’d recommend taking a look at the documentation and examples on GitLab. Amongst others, Jakub Jareš has also written an excellent series of articles on its use, which you can find on PowerShellMagazine.com

That’s it for this series of articles on the use of PowerShell and GitLab. Thanks for reading, and feel free to provide feedback.

Part 3 of this series on using PowerShell and GitLab CI gave us an initial insight into how to setup and run PowerShell code as part of a build script. During it, we were introduced to the YAML build file, .gitlab-ci.yml.

This article, and the following one will cover some of the gotchas I’ve encountered whilst getting to grips with this file, and how to get these working as you’d expect. Hopefully these will help save you some time setting these up.It will also touch on some oddities you might experience creating build scripts.

Overview

At this point, I’m going to continue to use the same project with the same job name, even though it will deviate from the original script we created earlier. This is to allow us to go through the situations I’ll mention slightly faster, and I’ll indicate for each example what the subject is about. Let’s begin!

Succeeds

Don’t use the TAB character

Unless you are using an editor which automatically translate the use of the TAB key into spaces, your scripts will fail. Use normal spaces.

Fails

The script below, whilst appearing indentical to the one that succeeds, uses table characters for indentation.

YAML

1

2

3

4

5

ps_helloworld:

script:

-.\HelloWorld.ps1|Out-File'c:\windows\temp\helloworld.txt'

tags:

-windows

Succeeds

YAML

1

2

3

4

5

ps_helloworld:

script:

-.\HelloWorld.ps1|Out-File'c:\windows\temp\helloworld.txt'

tags:

-windows

Watch Out For ScriptBlocks

The formatting for the use of a scriptblock can sometimes be a bit confusing. The final brace that closes the scriptblock should not have a ‘-‘ prefixing it. Additionally, if there is only one line of code within the scriptblock, it is not neccessary to use this prefix (but see below)

Fails

YAML

1

2

3

4

5

6

7

ps_helloworld:

script:

-For($i=1;$i-le5;$i++){

-.\HelloWorld.ps1|Out-File'c:\windows\temp\helloworld.txt'-Append

-}

tags:

-windows

Succeeds

YAML

1

2

3

4

5

6

7

ps_helloworld:

script:

-For($i=1;$i-le5;$i++){

.\HelloWorld.ps1|Out-File'c:\windows\temp\helloworld.txt'-Append

}

tags:

-windows

And if we actually look a the build output, we will see the details.
You might be quite correctly thinking that something looks a bit unusual with the output. This will be covered shortly……..

And a look at helloworld.txt to make sure it has worked correctly.

You Must Use ‘-‘ For Any Code Within Braces If It Consists of More Than One Line

If you have any code that surrounds itself in braces, such as a scripblock, function, or reiteration action, it will execute successfully without using the ‘-‘ character if there is only one line. However, two or more lines of code will fail.

Fails

YAML

1

2

3

4

5

6

7

8

ps_helloworld:

script:

-For($i=1;$i-le5;$i++){

.\HelloWorld.ps1|Out-File'c:\windows\temp\helloworld.txt'-Append

.\HelloWorld.ps1|Out-File'c:\windows\temp\helloworld.txt'-Append

}

tags:

-windows

Succeeds

It is probably best to start using ‘-‘ even with one single line within braces.

YAML

1

2

3

4

5

6

7

8

ps_helloworld:

script:

-For($i=1;$i-le5;$i++){

-.\HelloWorld.ps1|Out-File'c:\windows\temp\helloworld.txt'-Append

-.\HelloWorld.ps1|Out-File'c:\windows\temp\helloworld.txt'-Append

}

tags:

-windows

The script works, but the output on the screen is even stranger!

But the results are good….

Watch for error handling

Watch your syntax if you are wanting to use a Try..Catch block
Also, code within

Fails

YAML

1

2

3

4

5

6

7

8

9

10

11

ps_helloworld:

script:

-$ErrorActionPreference='Stop'

-Try{

-1/0

}

-Catch{

-Write-Output"Theres been a division by zero action"|Out-File'c:\windows\temp\helloworld.txt'

}

tags:

-windows

Succeeds

YAML

1

2

3

4

5

6

7

8

9

10

11

ps_helloworld:

script:

-$ErrorActionPreference='Stop'

-Try{

-1/0

}

Catch{

-Write-Output"Theres been a division by zero action"|Out-File'c:\windows\temp\helloworld.txt'

}

tags:

-windows

So the script has successfully executed. Again, the output on the screen is a bit strange, but let’s check the helloworld.txt file out.

However, the file itself has not been created, despite the fact that we know this should raise a division by zero error, which should mean the catch scriptblock is executed.

Functions Behave Differently Than You Would Expect

This final part is the greatest challenge i’ve come across so far, and deserves a bit extra detail. It will also be covered further in the next article in this series.

Here, we’ve modified our build script by adding a function which returns a string, and then output it to the same text file we’ve been using previously. If you want, you can verify the PowerShell code in the ISE, and also that the helloworld.txt file has been successfully created.

This would be put into our build script like this :

YAML

1

2

3

4

5

6

7

8

9

10

ps_helloworld:

script:

-functionHappyHelloWorld{

-$x="Hello world"

-Write-Output"I would just like to say $x"

}

-HappyHelloWorld|Out-File'c:\windows\temp\helloworld.txt'

tags:

-windows

And it builds successfully

So let’s take a look at helloworld.txt

This is strange!!! There is other items there that shouldn’t be. In fact, only the last line of the file should be there.

Let’s do a bit of detective work and see if we can get more information about what’s being returned. A good starting place would be to see if we are getting a multiline string or an array returned.

YAML

1

2

3

4

5

6

7

8

9

10

11

ps_helloworld:

script:

-functionHappyHelloWorld{

-$x="Hello world"

-Write-Output"I would just like to say $x"

}

-$result=HappyHelloWorld

-$result.GetType()|Out-File'c:\windows\temp\helloworld.txt'

tags:

-windows

So let’s take a look at helloworld.txt

This is real strange, an array is being returned. But we know that only a string is returned from the function. In the next article, we’ll dig deeper into what’s actually happening during the build process, which will shed some light on some of the strange things we’ve been noticing. Then we’ll also take a look at Lint, and how we can use it carry out checking of our YAML files.

In part 1, and part 2 of this series, we covered the installation and configuration of pre-requistites, setting up our GitLab account and initial settings, and finally getting our Windows Runner installed and operational.

Today’s, I’ll give an overview of how GitLab uses build configuration files, how it is constructed, and setup our project in preparation for getting our first build complete.

GitLab Build Conifiguration

Previously, I’d been using Jenkin’s for my CI system, and immediately noticed how build configuration is different. With GitLab CI, the build file itself, .gitlab-ci.yml, is actually part of the repository. No webhook setup is necessary.

We also now have capabilities, such as not only having a dedicated runner per project, but per branch of our project, which we can control and manage directly from our repo.

Create a Demo PowerShell Script

Let’s setup a really simple script which we want to pass through the CI Engine.

Log back into the GitLab environment

Open your project

Now launch the PowerShell ISE, and make the following script:

PowerShell

1

2

3

4

$x='Hello'

$y='Word'

$strHellowWorld="$x $y"

Write-Output$strHellowWorld

Save the file as HelloWorld.ps1 into the root of your repository (C:\temp\helloworld in my case)

Commit and Push to GitLab

We’ve added content to the repository, so lets go ahead and push the changes to GitLab.

Start a Bash session

Set your current current directory to the root of your repository

Enter the commands below

1

2

3

git add HelloWorld.ps1

git commit-m"Created initial HelloWorld script"

git push origin master

Now return to GitLab, and select Commits to confirm the commit has occurred.

Results

To the right of the commit, you will see a red ‘x’ mark. You might think that this is an options to delete the commit that has been made, but it’s something else entirely.

Click on the red cross. Looking further down the screen we see an error message.

Found errors in your .gitlab-ci.yml:
Undefined yaml error
.gitlab-ci.yml not found in this commit

As mentioned, .gitlab-ci.yml is the build file that stored within the repository itself. The commit has been successfully placed into source control, but because we enabled CI for this project, GitLab has also expected the build file to exist.

A Quick bit About YAML and gitlab-ci.yml

We’re now at the stage where we are wanting to design our build configuration file for the helloworld project.

Build actions for GitLab projects are stored within a file, .gitlab-ci.yml, which is placed in the root of a repository. This file uses YAML format, and describes the build actions and criteria. Those of you familiar with AppVeyor may be aware of this format of file, and the requirements it has to be successfully parsed.

As a newbie I struggled with this file, and it took me a whole day before I was able to get a successful build to occur (though it wasn’t helped by my build script being quite longer than our example). I’ll cover this file in a bit more detail, and the gotchas I’ve come across in a later blog. But for now, we’ll make a basic build file.

Building our gitlab-ci.yml file

Launch Notepad, and paste the following text into it:

1

2

3

4

5

ps_helloworld:

script:

-.\HelloWorld.ps1|Out-Filec:\windows\temp\helloworld.txt

tags:

-windows

Our configuration file consists of a label at the top, which identifies the name of the build, the script to be executed, and uses a tag of ‘Windows’ to identify the Runner to be used.

You might be wondering why I placed :

PowerShell

1

|Out-Filec:\windows\temp\helloworld.txt

in the build script instead of directly into the PowerShell script. The reason for this is simply to demonstrate how we can actually use PowerShell within our build script, and not just call PowerShell scripts.

Save the file as .gitlab-ci.yml within the repository, and return to Bash.

Add, commmit, and push this file to GitLab.

Results

Let’s take a look at the status of this build from GitLab. What’s particularly interesting here is that the actual command within the the script section of the build file is displayed. More about that later….

So our build has reported as being successfully, and if we take a look at C:\Windows\Temp on our build server, the file is there. Our first experience with GitLab CI is complete! 🙂

Coming Soon….

The next article in the series will cover some difficulties i’ve had getting to grips with GitLab CI, and how I was able to go from this :