Active forum topics

Use ISA (Proxynet) Proxies

UCT’s Campusnet proxies (running ISA) require NTLM Authentication, so that they can count usage against the right students’ quotas. NTLM Authentication is relatively non-standard, and rarely implemented in clients.

There are a few Linux applications which can connect to NTLM proxies directly, but most applications cannot. The solution is to use an “NTLM Proxy” which listens for requests from programs on your machine, and forwards them to the UCT servers with NTLM authentication done automatically. CNTLM is the preferred proxy.

OSX Users can use Authoxy for this. Setting it up isn’t described on this page, but you should be able to work it out from these instructions.

Cntlm in recent versions of Ubuntu

Cntlm is a proxy which performs the NTLM authentication for you, and passes on the requests to proxynet, correctly authorised.
It is the new implemented-in-C replacement for ntlmaps (described below). It is available as “cntlm” in Debian and Ubuntu.

Install the package

$ sudo apt-get install cntlm

Obtain the password hash of your domain password (you can put your raw password in your configuration file instead, but this is a little more secure). Your domain may be wf.uct.ac.za or WF — check which one works for you:

$ /usr/sbin/cntlm -u *YOUR_STUDENT_NUMBER* -d *YOUR_DOMAIN* -f -H

Copy the PassLM, PassNT and PassNTLMv2 lines from the output of that command (tip: use Shift-Control-C to copy selected text from a terminal)

Edit the cntlm configuration file ‘/etc/cntlm.conf’

$ sudo nano /etc/cntlm.conf

Paste the PassLM, PassNT and PassNTLMv2 lines command into the configuration file (tip: use Shift-Control-V to paste into a terminal), and edit it so that it looks like this:

If you see a screenful of errors, something is wrong, check your config (and see the debugging help below below).

Restart cntlm to apply the new configuration:

$ sudo service cntlm restart

Debugging CNTLM

CNTLM is quite fussy, it needs the hostname of your machine to match the hostname the proxy sees. You can get around this by telling it what hostname the proxy is expecting with the Workstation option in cntlm.conf. You can see the hostname the outside world sees, by running host 137.158.1.1 or dig -x 137.158.1.1 where 137.158.1.1 is your IP address.

Try putting the config snippet that the -M test gives you in your configuration file.

Configuring your system to use CNTLM (for GUI applications)

Now that cntlm is configured and running, we need to tell our applications to use it (alternatively, you can use the transparent setup described below; then programs won’t need to be told about the proxy).
Open the Network applet in System Settings, or simply click on the Home button and start typing ‘network’.

Click on the ‘Apply system wide’ button. If you take your computer off campus, change the method to ‘None’ and click ‘Apply system wide’. When you return to campus, change the method back to “Manual’ and click ‘Apply system wide’, you don’t have to configure the proxy settings every time.

Go to Firefox’s connection settings and select ‘Use system proxy settings’. Google Chrome uses the system proxy settings by default.

There appears to be a bug in recent versions of Firefox which prevents the browser from detecting the system proxy settings correctly. If you think that you are affected, try entering the proxy settings into Firefox manually. This does mean that you will have to change them in an additional place whenever you leave campus or come back.

Configuring environment variables (for command-line applications)

If you want command-line applications like wget or pip to use the proxy, you need to set some environment variables in your shell. These instructions assume that you are using bash (which is the default on Ubuntu); you should be able to adapt them to other shells.

You can also use the no_proxy environment variable to specify some domains for which you don’t want to use a proxy.

Note that editing this file will not affect your currently running shells — either close your terminals and re-open them, or set the variables manually in those terminals (by pasting in the lines above). You can test whether a terminal has the variables set like this:

echo $http_proxy

If this prints the proxy value, you’re good to go. If you get a blank line, something is wrong. You should be able to download a test file using wget:

wget www.google.com

Using the proxy with sudo

Now you have set up the environment variables for your user, but there is one more thing that you have to do so that you can use the proxy while using sudo — for example, if you need to install things from the internet through the proxy (Warning: installing everything through the proxy can eat through your quota fast — read the section about apt below for more information).

By default, when you use sudo none of your environment variables are preserved. You need to edit your sudoers file to make an exception for the proxy variables. You should never edit the sudoers file except by using the visudo command, which makes the process more secure:

sudo visudo

You should see a line in your file which says:

Defaults env_reset

Just above this line, add the following line:

Defaults env_keep = "http_proxy ftp_proxy https_proxy no_proxy"

Save the file. You should now be able to download a test file using wget through sudo:

sudo wget www.google.com

HTTPS

Please note that the campus proxies silently fail when attempting HTTPS over any port other than 443.
For now, you can remove the HTTPS proxy settings and use HTTPS without going through the proxies.

Applications which can access NTLM directly

You can configure Konqueror and Firefox to work with Campusnet directly by manually setting the Autoconfiguration file to http://www.uct.ac.za/cache.pac. This means that you have to change your proxy settings whenever you arrive on campus. This is a massive bind to do, but with the help of a Firefox add-on, you can make it slightly less painful.

Windows users are required to make a configuration change in Firefox, for NTLM authentication to work correctly. Linux users needn’t do this.

The only non-browser program known to work with NTLM Authentication is curl, which can be used as a wget substitute.

NTLMAPS

NTLMAPS is older than cntlm (described above) is more reliable, but slower and doesn’t support newer password hash formats that UCT now uses. (It’s packaged under that name for Debian/Ubuntu/most other distros).

apt and other package managers

LEG provides mirrors for a lot of Linux distributions. This means that you can install most packages (except those from non-standard repositories which are not mirrored, e.g. Ubuntu PPAs) from the UCT intranet, without using a proxy. If you find yourself needing to forward apt through campusnet to access the LEG mirrors, then something is wrong — contact the LEG admins or ICTS.

Warning: if you have a quota, you don’t want apt (or whatever package manager you use) to go through campusnet, as it’ll rack up quota usage very quickly. Make sure that you aren’t using ntlmaps/cntlm when you download packages.

If you don’t have a quota, you may find it useful to configure apt to go through the proxy so that you can use unmirrored repositories like PPAs. If you use apt on the command line you will need to set up the environment variables as described above. Whether you use a command-line tool or a GUI tool like Synaptic, you will also need to edit the /etc/apt/apt.conf file and add these lines (replace 3128 with whatever port your proxy listens on):

If you have a quota but you want to use PPAs, you can try using the no_proxy environment variable or the NoProxy setting in cntlm.conf to exclude the LEG mirrors. Make sure that your package manager is set up to use these mirrors!

A transparent proxying solution

The “transparent” proxy will pick up all outgoing port 80 traffic, and proxy it. Thus you shouldn’t need to configure any applications, they should just work as soon as you start the proxy.

We’ll use tinyproxy as our transparent proxy, and it’ll pass the requests to ntlmaps/cntlm. On Ubuntu tinyproxy is compiled with transparent support by default, but not so on debian. Debianites might have to recompile it…

# aptitude install tinyproxy

We use an /etc/default/tinyproxy script to insert the necessary iptables rules to intercept the traffic. Note that we only intercept off-campus traffic:

to channel traffic to *.uct.ac.za directly to the servers, avoiding the proxy server

to send all other traffic through to ntlmaps which is running (as described above) on localhost.

to make it easier for me to swith to different proxy settings when off campus.

I couldn’t get the transparent proxy to work…

OK - my setup:
Everything (firefox, thunderbird, …) is configured to connect through tinyproxy (except of SOCKS proxy, which I did not manage to compile into tinyproxy - but that’s fine for the moment)
The main changes I did were:

# Sending all traffic through to ntlmaps
upstream localhost:5865
## except of the following:
no upstream ".uct.ac.za"
no upstream "localhost"
no upstream "127.0.0.1"
no upstream "127.0.0.2"

I put these files in a separate file, called tinyproxy.conf.upstream.UCT

If you look at the cache.pac, there are quite a few exceptions - various .uct.ac.za domains that aren’t hosted on campus. Judging by IP Address is probably simpler (My iptables rule is currently doing that).

Once I configure CNTLM and tinyproxy I can add the PPA but once I try fetch updates from the off-campus repository I get the error: 407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )

Is there some way to configure apt to fetch the local updates bypassing the proxy (and thus the local transparent proxy as well) for updates hosted on leg - but to then use them for updates hosted on the PPA?

Your HowTo says I shouldn’t use cntlm with apt-getting, but I couldn’t set the universe and multiverse repositories without using cntlm, and without that, I couldn’t install Firefox, without which I can’t surf the web (Konqueror is broken in Kubuntu 7.10 with proxy authentication). So what to do?

After following the steps suggested here, I noticed that many websites’ layout was totally messed up. Any thoughts on why that might happen? I suspect something to do with the css files not getting through the proxy or something like that

I’ve also set up a LAMP server (linux, apache2, MySQL, PHP5). I need to install http_pecl into PHP.
When I try this I get a number of errors. The errors appear to be because I am unable to connect to the web from the command line.

I tried wget and it is unable to resolve the connection.
How do I allow the command line to connect to the net using the proxy.
errors from wget
—2010-05-26 13:23:30— (try: 2) http://pecl.php.net/get/pecl_http-1.6.1.tgz
Connecting to pecl.php.net|76.75.200.106|:80… failed: Connection timed out. Retrying.

On my computer both cntlm and ntlmaps won’t connect to the ISA proxies at UCT with my log in details. However, if I use another person’s log in details everything works fine. So if this isn’t working for you, it might just mean that ICTS just doesn’t like you either.

I’m a new staff member at UCT and I’m trying to configure my system and I am having trouble with the proxy settings. I’m running Ubuntu 11.10 on a Dell PowerEdge-R715 server. I followed the instructions on this page in relation to the cntlm, and it looked like it was installed properly. That is until I went to install a package in R which is not held in the local cran mirror site and I received the following error message:

The unable to connect to ‘wf’ on port 80 made me think that I need to do something else to the configuration or that I should also install and configure the tinyproxy. I did so and it also did not work. Downloading and installing this package was only a test. I really need to figure out this proxy server problem before I go any further on a more complex issue. Any suggestions?

on my laptop I also had to uncomment the Auth field and set it as:
Auth NTLM

Proxy campusnet.uct.ac.za worked for me. The weird thing is that I my other machine (my desktop) works well without the Auth field.
Maybe there is some MAC filtering going on, and known machines can use different authentication schema.

@Pedro:
I have also struggled to get R to go through cntlm +/- tinyproxy, and just figured this out:
What works for me is to set the proxy to “proxynet.uct.ac.za” before running R:

export http_proxy=http://proxynet.uct.ac.za:8080
R

Once inside R you can check that it is set correctly with Sys.getenv("http_proxy").

And to save some typing, one can define an alias in ~/.bashrc or ~/.bash_aliases like so:

alias rproxy="export http_proxy=http://proxynet.uct.ac.za:8080; R"

so that you can just type “rproxy” to invoke it with the correct proxy setting every time.

Hope that helps!

By the way, this also works in OS X.
If there any Mac users reading this who prefer to call their programs from the Applications folder in Finder, put a script like this into that folder:

export http_proxy=http://proxynet.uct.ac.za:8080
open -a R.app
exit 0

(The “exit 0” is to automatically close the terminal window that pops up.)
Save the script with the extension “.command” and make it executable by typing into a terminal, e.g., chmod +x rproxy.command. Then double-clicking it should open the R graphical interface with the correct proxy.

@Pedro:
I have also struggled to get R to go through cntlm +/- tinyproxy, and just figured this out:
What works for me is to set the proxy to “proxynet.uct.ac.za” before running R:

export http_proxy=http://proxynet.uct.ac.za:8080
R

Once inside R you can check that it is set correctly with: Sys.getenv("http_proxy").

And to save some typing, one can define an alias in ~/.bashrc or ~/.bash_aliases like so:

alias rproxy="export http_proxy=http://proxynet.uct.ac.za:8080; R"

so that you can just type “rproxy” to invoke it with the correct proxy setting every time.

(Note that, AFAIK, this setting for the http_proxy environment variable will persist until you start a new terminal session, so, e.g., don’t use apt-get straight after R without first opening a new terminal.)

Hope that helps!

By the way, this also works in OS X.
If there any Mac users reading this who prefer to call their programs from the Applications folder in Finder, put a script like this into that folder:

export http_proxy=http://proxynet.uct.ac.za:8080
open -a R.app
exit 0

(The “exit 0” is to automatically close the terminal window that pops up.)
Save the script with the extension “.command” and make it executable by typing into a terminal, e.g., chmod +x rproxy.command. Then double-clicking it should open the R graphical interface with the correct proxy setting.

Hey - so you typed “sudo cntlm -M http://google.com/”
but did you copy the response (those 2 lines between the — symbols) into your cntlm config file?
I think it’s the one in “/etc/cntlm.conf” or something like that (I’m on a windows machine atm)

Also, did you do something like “export http_proxy=localhost:3128” so set the proxy?