Posts

If you work for a corporate, chances are that you will need to deal with a proxy when it comes to connecting to the internet (or any computers that are outside the domain).

This can be very frustrating at times, because you now need to jump through another hoop in order to get your work done.

I’m writing this article in the hopes that it will help someone in the future, should they come across a similar problem or even the exact problem. This article covers setting up automated builds with TeamCity and pulling down a git repository from Assembla – all while behind a corporate proxy.

The corporate proxy that is used in my case is squid proxy (a common choice these days).

TeamCity Setup

The usual way of setting up TeamCity is allowing the server and the agent/s to run under a system account. This allows you to avoid password changing problems (should the password for the user that you set to run the server and/or agent/s, change), which is very common in the corporate setting (with a normal minimum password reset required every 30 days or so).

Following the default settings I came across the following problem with nuget restore. When running nuget restore from TeamCity it would always log that it was starting the download but then it would say that the connection was closed by the remote host. This error message was not very useful to me.

I searched around for about 2 days and found a couple of ways to run the TeamCity server through a proxy, but was not able to find a way to apply this proxy route to the agent. The agent is where we actually need the proxy information be applied because the agent is the one that is running the build steps (in this case, nuget restore).

I then came across this stackoverflow question: http://stackoverflow.com/questions/21043355 and this was where I found an answer to the problem. It’s still not ideal, because basically what it means is that we need to make the agent run as a user on the domain.

Through my searches I also came across this stackoverflow question here and here. On the second link, I posted my own answer. The details in the answer is what I ended up doing and this worked for me.

If it possible (as it was my case) – get a user uses password does not expire on the domain. If at first they do not buy into this idea, you need to explain to them that by not having this in place you could run into problems further down the line for your automated builds, since each month (or whatever the password change policy is), the builds will start failing because it does not have access to run under the user that you have specified.

Git Setup

What we would like to do is use ssh over port 443, since port 22 (normal ssh port) is blocked for security reasons. One of the main reasons that I can think of is that port 22 is also used for FTPs.

I learnt that this was possible when I stumbled upon this stackoverflow question here and in particular look the answer provided here and here.

However, due to Assembla’s lack of documentation for using ssh through port 443, this task turned out to be more tricky than explained in the answer mentioned above. Luckily, I got the information from one of the support guys and have provided the details for you below.

The hostname to use is altssh.git.assembla.com (this reflects to IP: 64.25.188.37)

As such, following the information provided in the questions above what we can do is add the following to our ssh config file (located at: ~/.ssh/config – if you are on windows, simply open git bash and navigate to that folder)

Once you’ve got this setup you can test the connection by running the following in git bash:

1

ssh-T-vgit@git.assembla.com

You’ll notice that it connects to the above mentioned IP address on port 443.

If you are lucky this will have solved your problem, if you are unlucky like me – you’ll get a connection timed out error.

Git Setup (Alternative 1)

If the above section doesn’t work, there is another option that you can try (which in my case also didn’t work and I needed to make changes to the proxy to allow this. You can see what I did for that in the section below titled: Proxy Changes)

You can try route the ssh traffic through the proxy using socat (a proxy client).

This requires you to download and install cygwin for windows (and also include the socat package – this can be down via the installer. If you need assistance with this, follow the guides on cygwin’s website)

There were a couple of methods that I came across (which I’ll list below) but by far the quickest way that I found was from this post here (i.e. Method 1)

Method 1

Following the link above from the post I mentioned you’ll notice that you need to make changes to the ssh config file. Here are the changes that I made:

Vim

1

2

Host git.assembla.com

ProxyCommand socat-PROXY:10.10.10.10:10.10.10.10:9418,proxyport=8000

Expanding the example

10.10.10.10 – this is the actual proxy address and will be different in your case

9418 – this is the port to communicate with on your proxy. In my case we are using 9418 (because this is how it was configured on the squid proxy – see this answer on stackoverflow for more information (it is important to make sure that the proxy is configured this way).

8000 – the port that the proxy communicates on.

Method 2

The other method involves running socat through git rather than running all ssh connections through socat. You can find information on how to do this here

Important note. Make sure that the path to the socat file exists. If you’ve installed cygwin on a 64 bit machine you likely have the files sitting under C:\cygwin64 instead of C:\cygwin.

You can place the gitproxy.cmd file in any location, all that you need to do is when you run the command:

Vim

1

git config--global core.gitproxy gitproxy.cmd

you need to provide the full path to the file. For example:

Vim

1

git config--global core.gitproxy"C:\MyAwesomeFolder\gitproxy.cmd"

There is another example using socat from an .sh file which you can find here. The same information applies to this one. Make sure that cygwin’s bin folder is in your path for this to work (or change the socat in the file to be the full path – like with the previous example).

Proxy Changes

Finally, we managed to get the proxy and the server to play nice with each other.

Our investigations lead us to this web page here. One this page, it shows you that you can use ssh to ssh into another server as a proxy command. This basically means all we need to do is ssh into the proxy server from our server and viola! (Except not as easy as “viola!” as there are settings that need to be configured)

One of the most important configurations required for this to work (as you might notice from the web page linked above), is the presence of netcat (i.e. nc). Once you have installed (or confirmed that) netcat (is on) the server you will need to make changes to the ssh config file.

Vim

1

2

3

4

5

6

Host git.assembla.com

Port443

Hostname64.250.188.37

ProxyCommand ssh-quser@10.10.10.10nc%h%p%r

Usergit

IdentityFile~/.ssh/myprivatekeyfile

Expanding the example

You will notice that in our ProxyCommand we are now running the ssh command to connect to the proxy server directly using the user “user”.

By doing this when we run the test (namely running

Vim

1

ssh-T-vgit@git.assembla.com

) we will see that it asks us for a password. This password is the password for the user that you have provided in front of the @ in the ProxyCommand line.

You will notice that there is a -q after the ssh in the ProxyCommand line. This suppresses the “Killed by signal 1” message from the terminal of the ssh connection. You’ll find more information about this here.

Avoid using the password

To avoid using a password, simply create an ssh key for the user that you are using to ssh into the proxy server. Copy this key to the server that you are running the command from.

Using my previous examples as reference, create a key on 10.10.10.10 (the proxy server) for user “user”. Now copy this key to the server that you are running the “ssh -T -v git@git.assembla.com” command from.

This newly generated key must also be added to the authorized_keys in that user’s .ssh folder.

Once you have done this you will need to append your ssh config file with the following information:

Vim

1

2

3

Host10.10.10.10

Useruser

IdentityFile~/.ssh/newproxykeyfile

This will make sure that when the ProxyCommand is run, the correct key (IdentityFile) is used for that host.

Final changes for TeamCity required

After you’ve gone through this entire ordeal, you might still be experiencing a problem with using the VCS from TeamCity (as I did). You might run into a situation where you receive a “Connection timed out” message when TeamCity tries to pull from your repository using the VCS settings.

Why this happens is beyond me. I can’t seem to find any information regarding it on the web.

I was able to come across the following link, which details how to change the TeamCity server proxy settings (via the environment settings). If that link helps you, that’s fantastic! Unfortunately, in my case it did not work.

There was another setting which I found that could be tried (but this alsodidn’t work). You can find the details for that here (there is an answer on this question which links to the TeamCity page which will explain how to make the required changes).

Finally, what I decided to do was to create a new build step (in TeamCity) that would run a “git pull origin master”. This is the first build step of course. This runs in a “hard-coded directory” which is then referenced as the path in all the other build steps.

If all of the above does not work for you and you would prefer not to specify the checkout folder directory in each build step after, I have some bad news for you. You will not be able to do a checkout using the git protocol. You will need to use HTTPS, which is explained in my final section.

This is of course to my knowledge. If anyone knows how to get it working beyond what I’ve mentioned in this post, please comment and let me know what you did to get it working.

If nothing above works for you

If all else fails (from the above) and you are unable to modify the proxy to allow the connections, there is one last saving grace.

This is simply to use https to clone the repo.

Of course, it’s still not just that simple, since we are behind the proxy. There is one additional change that you will need to make before you can clone out https.