Monday, July 04, 2005

Boy, I sah-crewed da pooch with that offhand Turkish "i" comment. A'ight. I know how to back-pedal and regroup and try to come up with a lucid, compelling defense of what I was thinking...

Nope, can't do it. I typed and I typed and I typed and it just sounded like myopic drivel that I only half-believed in myself. Maybe even I'm succumbing to fatigue.

The issue is that I'm frustrated by the loss of mainstream features that have to be cut in order to code and test against hard international features that will have a marginal return-on-investment at the expense of cutting something that Fortune 500 companies would be delighted to see and deploy. Our competitors don't sim-ship in all those languages and it is an opening for them to establish competitive ground.

Return-on-investment. Balancing features vs. international markets is sort of a knap-sack problem, except when done right, you empty the sack and get increased deployment and adoption, which translates into cool, crisp cash. That problem is not best represented by a vector of "i"s but something much more complex that I doubt any of us will agree upon. Personally, I'm fighting right today for the decisions that will have Microsoft actually shipping product NOW and putting down a foundation to grow into for the next decade.

I work in OSS projects, so in a way I am a competitor. But we all that turkish 'i', just like any other "doesn't work on my obsolete (win98,OS/360,VMS) box in french canadian when in low power mode", or "fails when AOL is the dial up ISP", or even "doesn't work on feb 29, but only in GMT0BST tz". These little config problems are the bane of everyone's life. So yes, lets think out the box about fixing it. Why dont we make turkey fixing its case logic part of its EU accession rules, you know "adopt the (€) euro currency, stop torturing suspects and make tolower("I")=="i".

Then we can take on the fact that different places switch to summer time differently, sort out the india/pakistan border so that the world clock applet can have clicking turned on again, and abolish USLetter paper in favour of A4.

That leaves finding whoever invented the Euro currency symbol and giving them a good kicking for using a symbol that wasnt in the known alphabet, when there were many symboles unused except in Perl that could have been chosen instead. timezones,

I'm not in 24, but I'd like to see that list, too. I think this Turkish i argument is either a huge straw man (what's the real reason you can't ship?) or just a poorly-chosen example of things that cause dev schedule randomization. A better example might be all the wasted dev time spent triaging bogus Prefix bugs.

Drew is right, triaging prefast/prefix bugs is a huge time sink. Quality gates, while a noble idea, are a huge time sink. Ever notice how all the quality gates tools are written by people who've never produced a line of shipping code? It's often MSR twits who someone convinced a VP or two their analysis tool will save the company. Then point to some past security bug, show how their tool actually would have caught it, and then we have more redtape, when the problem wasn't really the lack of their tool, it was just having a sloppy dev in the first place.

Hang on. I'm not saying that Prefix/Prefast suck. I actually like them. I'm just saying that recently there seem to have been a lot of non-bugs found resulting in way too much triage time. It's the high number of false positives that I have a problem with. Seems to me the root of the problem is that the bug-reporting threshold is just set wrong.

The idea of prefix/prefast is good, but the implementations are weak. And getting worse. This constant holier than though crap from MSR gets really tiring and costs a lot of money. Fine if someone wants to support the welfare state in MSR, but please stop unleashing their half assed work on the rest of us.

And why do we bother fixing all of those bugs? We could ship with way more bugs. People expect us to ship buggy software anyway. You know that only a few percent of the people are going to run into some of those edge cases. We could get way more features done if they didn't really have to work. Don't get me started on the subject of all those Watson bugs we have to fix.:-)

"And why do we bother fixing all of those bugs? We could ship with way more bugs. People expect us to ship buggy software anyway. You know that only a few percent of the people are going to run into some of those edge cases."

Sure - except that in this case, the bug's effect will be concentrated in one group of users. It doesn't matter if that bug cluster affects one nation, one gender or one industrial market segment; it's a bad idea to start treating a subgroup of your customers as if they deserve a lower standard of quality than the rest.

No, let me rephrase that - it's a marginally bad idea if you've already determined that subgroup merits less work on your part. At that point, it's merely a business decision.

The culpably stupid point comes before that, when you start using subgroups of your customers as places to hide your unfixed bugs. That's when the rot starts to set in.

"It's the high number of false positives that I have a problem with. Seems to me the root of the problem is that the bug-reporting threshold is just set wrong."

Is it the bug-reporting threshold -- the measure of what constitutes a valid bug? Or is it the use of raw bug numbers as a measure of a tester's productivity?

During my time as a tester at Microsoft, my group started out with a fairly complex find/fix/priority calculation to measure tester productivity. If you turned in fewer bugs, but with higher average priority and/or a higher fix ratio, you tended to get better pay than someone who churned out an endless stream of Pri 3s.

Over time and a few management changes, that devolved back to the old standard - more bugs = mo' better, so long as you met a minimum find/fix ratio.

Poorly chosen bugs can be fixed with improved tester training and good guidance from the product PMs. Poorly chosen quality standards require leads and managers to think a bit deeper about how they compensate their reports.

Disclaimer

These are sole individual personal points-of-view and the posts and comments by the participants in no way represent the official point-of-view of Microsoft or any other organization. This is a discussion to foster debate and by no means an enactment of policy-violation. These posts are provided "as-is" with no warranties and confer no rights. So chill. And think.