Ramblings from the creator of HomeSite, TopStyle, FeedDemon and Glassboard Android.

Friday, June 09, 2006

Microsoft, Please Fix this WinINet Bug!

Recently a few FeedDemon users reported CPU spiking when updating their feeds. After finding a feed which triggered the problem, I debugged my code and discovered that the CPU spike was actually happening in Microsoft's WinINet.dll (a Windows library which provides Internet-related functions). Specifically, the CPU load was jumping during a call to InternetReadFile - and the load never decreased the whole time FeedDemon was running. Not good.

Googling for info turned up this Microsoft KB article, which describes the problem as a bug in InternetReadFile. Luckily it isn't a bug that many people will run into, since it requires a specific situation involving "chunked encoding" and a server not closing a connection correctly. But for those who run into this bug, it's definitely annoying and it requires a solution.

The article mentions a possible workaround: use HTTP/1.0 rather than HTTP/1.1. Sounds simple enough, but another Microsoft KB article explains why that won't work. In short, the only solution is for customers to change their Internet settings, which is a PITA since we'd obviously rather resolve the problem without requiring any user action.

Now, I'm not going to criticize Microsoft for the fact that this bug exists - as much as programmers try to find and kill all bugs before a product is released, they're still a fact of life. Bugs happen.

But I am going to criticize them for how long this bug has remained unfixed. The KB article is dated October 2, 2003, so they've officially known about this bug for almost three years. This is a serious bug - it causes applications to stop responding or over-utilize the CPU, leading to crashes or decreased performance. In other words, this bug makes Windows developers look bad.

Microsoft, please fix this bug. You ship WinINet.dll with Internet Explorer, so you could include the fix with IE7. Windows developers need to know that they can rely on your APIs, and long-standing critical bugs in your APIs make us leery of relying on you.

Update: I just confirmed that this bug still exists in Windows Vista Beta 2.

BTW, don't even think about trying to change the user's HTTP setting. We tried it for a while and (upon reflection, rightly so) got spanked for it. It apparently breaks Windows Update under certain circumstances and can interfere with intranet logins.

I didn't. That upgrade turned off (in code) the "Always use HTTP 1.1 even if the program is saying to absolutely, positively, definitely, without fail, use HTTP 1.0" registry setting but a few people reported problems because of it. You can search our forums for more details (I'm not going to link to anything because some of it is somewhat less than complimentary :-)).

My research suggested that it would be safe to do this but I included a debug switch to disable it. In hindsight, I should've done it the other way around: leave the problem active and tell people how to fix it (manually) if it surfaced. It was partly a psychological issue since I came across it almost immediately after 2.1 had been released and I was frantic because I thought it was because of something I had done.

AFAIK, there's no way to fix this other than to ask people to manually turn off the HTTP 1.1 setting. Luckily, I've never seen it again.

Yeah, I've been having those kinds of issues with the latest build of FD. It not only spikes to 100% - but 100% across all four of my processors (double core Intel with HyperThreading). Not to mention, the update all feature "once an hour" doesn't seem to be sticking, but that may be an unrelated bug report.

Nick, we're taking a look at this. WinINET has undergone major updates for IE7, so it's interesting that you're still hitting this.

Do you have any repro URLs which consistently reproduce this problem? In working on Fiddler (http://www.fiddlertool.com) I haven't encountered any major sites which prematurely close chunked bodies, so it would be great to understand where you're hitting this problem.

Nick, I am so sorry this thing has caused you grief! For Vista, and going out with IE7 on XP/Win2K3, we have fixed a LOT of WinInet bugs, including a major cleanup of the cache.

As you noted, this particular bug can be difficult to reproduce, so I really appreciate the URLs. I reactivated the bug. Hope we can ask you for some help if we get stuck on it again.

If you're using WinInet or WinHTTP in your app, my team would be grateful if you could take some time to send us your input. Check out our blog for more on that.

You can also file bugs against WinInet (or any other of our components) via our connect site at http://connect.microsoft.com/wndp. Well-triaged bugs are always gratefully appreciated. For http client issues, it helps to make sure you can telnet on port 80 to the destination before shooting the WinInet messenger.

Long time ago I asked you if you were planning better support for Firefox, i.e., not to rely on Internet Explorer for displaying feed content. You said you were considering it, but it has never happened. Obviously, as pointed out here, the Internet Explorer is solution is not free of bugs; on the contrary, for years it has caused people a lot of grief without any solution to look forward to.

I hope you are still considering implementing Firefox as an alternative to IE.

Truth be told, though, the embedded Mozilla engine is far more buggy than IE, which is why it's an unsupported feature. I may be annoyed at MS for the WinINet bug, but regardless, they've done a far better job than Mozilla.org at making it easy to embed their browser.

I can't say for sure, since I'm not an expert in server behavior, but my guess is that the RFC2616 violation is happening because meyerweb has, of late, been just terminating connections without warning. I'll get half a page to load, and then it just stops. If the same thing is happening with RSS feeds, then I'd guess that the termination is so abrupt no terminating 0 is emitted.

I think this might also be why Safari keeps giving me "lost network connection" error messages, though again, that's a guess. And not somethinig you Windows guys probably care much about anyway.

I would definitely do something different with handling, EricLaw, though I'll put in a plea to PLEASE keep things in the realm of generating an error, even a nice descriptve one that users could share with site owners. And not spiking the CPU.

Nick, sounds like Mozilla.org is behind the curve on a stable embedded capability--how about Opera? I've grown to really like Opera, very fast and clean--seems like approaching Opera (even to suggest they provide this capability, if they don't already) would give customers more options.

...and thanks to both Nick and Microsoft for hopping on this so fast!

(I have also experienced the CPU spiking periodically, but it seemed to "get better" when I uninstalled Windows Defender Beta 1--WD Beta 2 seems okay as far as the CPU spiking goes. Don't know if there's any "real" relationship between WD and the CPU spiking, but I did find the Beta 1 problematic.)

The meyerweb feed may have given some folks grief but it was helpful getting to the bottom of this thing. Definite bug in the parser code handling improperly formed chunked responses. Fix is pretty straight-forward and has FeedDemon running. Nick, can you please shoot me an email? We saw something else, while looking at this, which we wanted to run past you. Thx! -David

Just to close the loop on this thing, we are pretty certain we've fixed the underlying bug. Our WinInet team got to the bottom of the bug before the sites Nick referenced made their changes. I doubt the fix made it into IE7 beta 3 on XP, but it should be in the final version when IE7 releases and should be in RC1 of Windows Vista (unless something unexpected happens).