OK now we get to the disclosure part, and I'm rather embarrassed by this. I asked the question before (it was 2003 and I was asking about WWW::Automate) and apparently I got a working answer, but I got it via someone's scratchpad and have now mislaid it. See this node.

I don't have time to play with this tonight, sorry, but one of the first things I'd do is try the login in firefox with the "Live HTTP Headers" extension turned on. That might give some insight into what's going back and forth.

Notice that your script is somehow sending you to a page in "mydetails", wich doesn't happen with the normal login process (take a look at the LiveHTTP log, you are not at any point being redirected to any page on mydetails directory).

So, for some reason, you are trying to post your data to a page that doesn't support the POST method, wich I think is done by some kind of "protection" against this kind of method.

I've tried playing with WWW::Mechanize once and I've found out that while I could easily use it to send an email on my site contact page, I couldn't use it to to manipulate other sites.

What seems to be the problem to me is that I think the http standard calls for Location: redirected requests to be in the same method as the original call (if a POSTed page redirects to another, that page should also be POSTed) - this, I think, is what WWW::Mechanize does - but not what Firefox and other standard browsers are doing, a behavior that the site developers are taking into account (even though they should be using a Status: 303 See Other, not a 301)... What you want to do is load up the cookie_jar with the authentication information, then request the particular pages you're looking for - you're falling to an error because autocheck is set to on, and when Mechanize POSTs to a page that expects a GET, it checks to see if it worked, realizes that it didnt, and all goes ka-ploowey. So... what you need to do, is authenticate yourself, like you wonderfully did (but with autocheck off)... then go ahead an request the user-particular webpage from which you wish to pull data: for instance, this modification of your code will authenticate you, and then pull up the 'mydetails' page:

Now, there's no reason to parse all that funky javascript - lots of folks have js turned off in their browsers - if the guardian didnt allow these people to browse their site, they would be losing a good portion of their readers - all that js hashing is for added security, but isn't required - as the above code demonstrates.

I hope this helps - I mean given your request, I think this is what you're looking for. And you should give yourself a pat on the back - you were like 98% right already!