I am currently trying to log into a site using Python however the site seems to be sending a cookie and a redirect statement on the same page. Python seems to be following that redirect thus preventing me from reading the cookie send by the login page. How do I prevent Python's urllib (or urllib2) urlopen from following the redirect?

You don't seem to be using redirect_handler = urllib2.HTTPRedirectHandler() in the example at all. Were you going to show a second example?
–
Ehtesh ChoudhuryAug 16 '11 at 21:13

You are correct, I'm not using the redirect_handler. Instead, I created my own redirect handler. I will edit to remove.
–
popeAug 23 '11 at 4:38

Why is it you do not need to instantiate the MyHTTPRedirectHandler, but rather pass the class into the build_opener() method?
–
BenjaminJan 9 '12 at 20:10

From the documentation: handlers can be either instances of BaseHandler, or subclasses of BaseHandler (in which case it must be possible to call the constructor without any parameters). Since MyHTTPRedirectHandler doesn't have a constructor with any arguments, I can pass it in as is.
–
popeJan 12 '12 at 1:43

If all you need is stopping redirection, then there is a simple way to do it. For example I only want to get cookies and for a better performance I don't want to be redirected to any other page. Also I hope the code is kept as 3xx. let's use 302 for instance.

You could try calling urllib2.build_opener(handlers) yourself with a list that omits HTTPRedirectHandler, then call the open() method on the result to open your URL. If you really dislike redirects, you could even call urllib2.install_opener(opener) to your own non-redirecting opener.

You could try calling urllib2.build_opener(handlers) yourself with a list that omits HTTPRedirectHandler, then call the open() method on the result to open your URL. Well, docs for urllib2.build_opener() say this Instances of the following classes will be in front of the handlers, unless the handlers contain them, instances of them or subclasses of them: ProxyHandler, UnknownHandler, HTTPHandler, HTTPDefaultErrorHandler, HTTPRedirectHandler, FTPHandler, FileHandler, HTTPErrorProcessor. It looks like ommiting HTTPRedirectHandler won't work...
–
Piotr DobrogostApr 1 '11 at 17:57

EDIT: If you have to deal with quirky web applications you should probably try out mechanize. It's a great library that simulates a web browser. You can control redirecting, cookies, page refreshes... If the website doesn't rely [heavily] on JavaScript, you'll get along very nicely with mechanize.