This is fixed in r81636, r81637, r81638, r81637.
Just like the Digest Authentication, the urllib2 will try for 5 times before failing for any authentication failure ( It won't lead to recursion).
I had a thought that Basic Authentication need not be retried (no where in the RFC it mentions about it), but I found some references that clients do present the dialog a couple of times for retry on wrong authentication, so we are going for the same.
This is tested with actual resource setup. Closing this issue as this bug is fixed.

Why would you waste the time and resources to test 5 times for a known to be wrong credential ?!
This is not like in a browser where the user is presented with a dialog box 5 times and he/she can try different credentials.
I don't see the point...

The point of retry is for auth failure, which can happen due to any reason. It is not just for a wrong password.
I was thinking just doing a single verification and failing, but did find some resources (not just browsers) which adopt this approach of retry.
BTW, Basic auth is gone now and Digest auth has some recommendations so I found its fine to do the same way as Digest auth does.

I would like to point out that this is not going to work if someone visits more than 5 sites with the same authentication manager. This would have to be documentated, at least.
We could fix this by putting the retry counter in the HTTPPasswordMgr; it is not hard to put in an extra field in the password database with the retry counter. See also my remarks in issue8894

+1 for putting retry control on the password manager. Probably the default should be set to 1. If I understand correctly, it is the password manager that would be prompting the user for a new password, if someone chose to implement such a password manager.

FYI:
The regression was introduced in 2.6.5 with issue3819 .
This also causes problems with Mercurial pushing to google code - see http://mercurial.selenic.com/bts/issue2179 .
IIRC google code redirects to an url which then in turn requests authentication. I'm not sure how that will work with a retry count of 1. The funny and inconsistent thing is that it apparently would work if they replied 403 instead of 401. "durin42" from google/mercurial knows the details.
(It seems suspicious to me that there are code paths where the retry counter is reset. Doesn't that mean that a remote server will be able to cause endless recursion? Shouldn't there be some kind of hard upper limit on the number of retries?)

I work in sidux and my Mercurial currently doesn't work. The python version already contains the fix for this issue (revision 81637) and it crashes Mercurial ("authorization failed") whenever a command involves more than 5 requests to the repository. I fixed it by resetting the retry counter upon successful authorization (see patch). Maybe this helps someone in a similar situation. The patch was made against trunk using "diff -u".

I've also run into this problem after upgrading to Python 2.6.6. My code, which uses the same HTTPBasicAuthHandler instance for many requests to the same server, worked correctly with Python 2.6.2 and broke with 2.6.6. It would be great if zenyatta's patch to fix the regression was included in 2.6.7.
Also, unfortunately NEWS.txt doesn't mention this change at all.

I checked in a modified version of reset the retry count for respnse code !=401 in the following checkins:
r84323 (py3k)
r84324 (release21-maint)
r84325 (release31-maint)
Unfortunately, we wont be able to patch the 2.6.x release. You might want to use patch which is attached or fix it at mercurial end.
Another issue, Issue9639 was fixed to reset the retry on successful response.
I am going to close this issue, there was a minor discussion in the middle if 5 retries in worth it, but it looks like it is (msg107261).

Senthil, can you tell us why this fix is correct - and convince us that it is the Final Fix for this issue? Not because I don't trust you, but because this issue has a bad track record.
Some comments/questions to this patch:
Why do 401 require such special handling? Why not handle it like the other errors?
How do this work together with http://code.google.com/p/support/issues/detail?id=3985 ?
Detail: I'm surprised you don't use reset_retry_count() - that makes it a bit harder to grok the code. And the patch doesn't reduce the complexity of the code.
But ... I really don't understand ... .retried is a kind of error counter. Why do we reset it on errors? I would expect it to be reset on success ... or perhaps on anything but 401, 403 and 407. Or perhaps it should be reset whenever a new URL is requested.

On Thu, Aug 26, 2010 at 3:24 PM, Mads Kiilerich <report@bugs.python.org> wrote:
> Mads Kiilerich <mads@kiilerich.com> added the comment:
>
> Senthil, can you tell us why this fix is correct - and convince us that it is the Final Fix for this issue?
Hello Mads, this may not be be final fix. I just adopted what was
already done for Digest Auth for the Basic Auth, it could be an
interim solution that avoids recursion problem.
I agree with you respect to the other error codes, there is already
another bug open to handle this. The reset counter is reset on success
too, in another part of the code. I shall respond to the other
questions a bit later (1 week, I am kind of busy at the moment) and
lets go for the best fix as discussed.
Thanks,
Senthil

On 08/27/2010 03:47 AM, Senthil Kumaran wrote:
> I agree with you respect to the other error codes, there is already
> another bug open to handle this. The reset counter is reset on success
> too, in another part of the code.
FWIW: From Mercurial it is our experience that the reset counter only
gets reset on errors, never on success.

I think there's a much simpler solution to this ticket than the retry logic that's currently in place.
The code originally avoided the infinite recursion by checking to see if the previous request had already submitted the auth credentials that would be used in the retry. If it had, it would return None. If it hadn't, it would add the auth credentials to the request header and the request again:
if req.headers.get(self.auth_header, None) == auth:
return None
req.add_header(self.auth_header, auth)
Then, to fix #3819, it was changed. Instead of calling add_header, it called add_unredirected_header:
if req.headers.get(self.auth_header, None) == auth:
return None
req.add_unredirected_header(self.auth_header, auth)
This caused the loop because the auth creds were going into unredirected_hdrs instead of the headers dict.
But I think the original logic is sound. The code just wasn't checking in all the headers. Luckily there's a get_header method that checks both for you. This one-line change should fix the issue:
if req.get_header(self.auth_header, None) == auth:
return None
req.add_unredirected_header(self.auth_header, auth)
I think this fix is cleaner and makes more sense, but I'm worried I might be missing something. I don't fully understand the distinction between headers and unredirected headers. Maybe there's a reason why the code isn't checking in unredirected headers for the auth header.
I'm attaching a patch. I'm new to contributing to python so I apologize if the format is wrong.

Thanks Sam! I can confirm that your patch is working well with basic auth. I have not tried other authentication methods, though. I'm not sure about the unredirected headers, but your fix looks like the cleanest.

This bug has been open for a while and I had lost sight of it. Upon prompted recently, I dug bit
into history and could think of a good solution.
A brief history.
1. The bug "maximum recursion depth exceeded" when doing Basic Authentication was introduced in
5f9939af2f71 (issue3819). I verified it by doing "hg update 5f9939af2f71" and running basic
authentication test against a server.
2. This was 'fixed' with giving an upper limit to retries and doing it similar to Digest
Authentication as part of this ticket and it is case recent current tip
(acb30ed7eceb when I ran the test).
However, this fix is not satisfactory for multiple reasons like fix is not satisfactory and not
having concrete reasons for doing retries for Basic Auth and why 401 error should be considered
special. I agree with this, this fix was more like to avoid recursion
and throw the correct HTTPError.
Also, it is important to note that, before a regression was introduced in r59118, the prior
revision ( hg update 5f9939af2f71^ -which led me to c8ef906b11b8) had correct behavior, where
HTTPError was thrown upon invalid user:pass immediately.
So, going back to that behavior was the right thing to do.
Sam Bull's suggestion and patch looks right to me. It is correct to verify the un-redirected hdrs
too when checking for auth and go with the behavior as it in c8ef906b11b8)
I have improved upon the patch for 3.5 and added tests which were not present too. I think, this
should go in all active versions of python as there is no api change or behavior behavior, but
improvement in way HTTPError is thrown.