A HTTP Detective Story by Eric Lawrence

As a little kid, my dad read with me a lot; we usually read detective stories. While of questionable literary merit, those books developed in me a burning desire to figure stuff out, to pull back the curtain, to understand the mysterious. I still maintain that curiosity today-- I joined the IE team because I wanted to learn the browser inside out, and I developed Microsoft Fiddler to expose the secrets of HTTP in a user-friendly way.

I recently came across a bug in the bug database for IE7 which was intriguing— logins for Internet Explorer 7 and IE6 on XPSP2 had started failing for subscribers at a popular magazine’s website. The site didn’t have any recommendations on how to fix the alleged “cookie flaw in SP2”, though the issue was discovered by their users during the SP2 beta.

Now, as most of you know, XP SP2 introduced major security improvements for users, while causing some issues for web developers. Before we shipped SP2, we fixed a ton of compatibility issues to ensure that IE continued to work on the broadest range of sites without making security sacrifices. It looks like we missed a compatibility break; this bug still hadn’t been resolved, so I dove in.

I fired up my trusty HTTP debugger, Microsoft Fiddler. Some developers in the audience might be scratching their heads-- I’ve got the full source for IE on my computer, so why start by looking at the traffic? Simple—it’s usually the clearest way to see what’s really going on. HTTP is beautiful that way— if you can see both the traffic of a working scenario and the traffic from a broken scenario, you can resolve most problems of this nature.

The site noted that subscribers using Mozilla FireFox or Opera could log in just fine, so I hooked FireFox up to Fiddler and watched the traffic flow by. Nothing seemed amiss, and the login worked as expected. Okay, now to try out IE on XPSP2… I performed the same login procedure, using the same credentials, and… hrm. That’s odd. The login silently failed and I was left sitting at the “Please sign in” button on the homepage. Very mysterious-- my curiosity was piqued.

I started by filtering the Fiddler session list to ignore all of the image and CSS requests, since they’re unlikely to be related to login problems. This left two HTTPS pages and two HTTP pages.

Using the Microsoft WinDiff feature inside Fiddler (select any two sessions, right-click and choose WinDiff from the menu), I found the body of the POST was identical in both cases, so that wasn’t the culprit. I also determined that there are no extra cookies being set or sent to FireFox, so the magazine’s hypothesis that this bug is caused by cookies is rejected.

The only cookie being passed around is a standard ASP Session ID cookie; I eventually decided that this cookie was probably associated with a serverside session variable set inside of auth.asp, e.g. IsLoggedIn.

I carefully looked for differences in the headers… Nothing jumped out at me. On a hunch, I added the following rule to my Fiddler Rules file:

staticfunction OnBeforeRequest(oSession:Fiddler.Session)

{

oSession.oRequest["User-Agent"]+="Gecko";

}

This rule will append “Gecko” to the browser user-agent header. I used Fiddler’s menu options to clear my cache and cookies and then retried the login using IE. And this time, login succeeded. What?!? That can’t be right. I disabled the rule, cleared the cache and cookies, and tried it again. Login silently failed. Hrm. Let’s try something else:

staticfunction OnBeforeRequest(oSession:Fiddler.Session)

{

oSession.oRequest["User-Agent"]="Opera/8.0(WindowsNT5.1;U;en)";

}

Login was also successful using “Opera” as the user-agent string. Interesting. Someone doesn’t want IE to log in. But why?

As with many mysteries, even though I thought I had collared the culprit, it turns out that something far more interesting was afoot. A single fact kept tickling at the back of my mind… The bug says this site used to work before XP SP2 was released. How could that be? I don’t have an unpatched copy of IE handy, but I might be able to fake it…

Login still failed. There must be something else that differs… I then recalled that I’ve got a rusty old Windows 98 image on a Virtual PC. I fired up the Virtual PC, pointed the proxy configuration at my XP SP2 box running Fiddler, and ran the login. It worked. Bingo.

With the click of a mouse, I performed a WinDiff of the Windows 98 login traffic against the XP SP2 login traffic… And I had my man.

This violates the RFC which states “Clients SHOULD NOT include a Referer header field in a (non-secure) HTTP request if the referring page was transferred with a secure protocol.” We fixed a bug for XP SP2 where if a HTTPS page 302 redirected to a HTTP page, the Referer was incorrectly being sent.

Could it be so simple? I added the following rule

staticfunction OnBeforeRequest(oSession:Fiddler.Session)

{

oSession.oRequest["Referer"]="https://example.com/loginverify.asp";

}

And login now worked on Windows XP SP2.

Wow, that’s really silly. Rather than developing a real security mechanism, they’re using a crusty old bug to check to see if the user is coming out of the secure login page, under the assumption that if they’re getting to Auth.asp from loginverify.asp, the login info must be correct!

It’s been said before, but I’ll say it again… never fully trust header data coming from the client, including the Referer. It’s horrendously insecure; as we’ve just shown, it can be bypassed in a single line of code.

Without ever looking at the source for IE or the source for auth.asp, I was able to deduce: