Thursday, June 29, 2006

Ethics

AWB has started an interesting conversation about work and research and personal and professional ethics, and I wanted to take the luxury of my own blog to expand a little on something I said in comments over there. (so go on over and get caught up. we'll wait. back? Ok, good.) One of the things I love about my profession is our set of professional ethical guidelines. (that's a reference to the American Statistical Association; although public health has its own professional society, they have no official ethical guidelines, which I find weird and troubling and is the topic for a whole 'nother post) And right there in section II.A.2. (" Guard against the possibility that a predisposition by investigators or data providers might predetermine the analytic result.") is the reason I take it so personally when someone jokes about 'lies, damned lies, and statistics.' Yes, as another commenter pointed out, this is a crazy ideal to aspire to (I kept trying to re-write that in my head as to which to aspire but it just sounds weird) but for me it's this great black and white rule to fall back on when reporting results to superiors. Some MD asks me to go back and try something else, look at the analyses this way, tweak the data that way (sometimes in ways that feel obviously manipulative, sometimes in more nefarious ways that just seem like legitimate data digging) and I can play the sorry-m'am-just-following-orders card. Numbers are numbers and they either say something or they don't. Yes, they can be interpreted five gazillion different ways, but the numbers themselves never lie, only people do. The Truth of the numbers sets the perfect bar for me to reach for in my interpretations. When I am being the best statistician that I can be I am presenting my analyses and my results and my interpretations in such a transparent way as to be both repeatable by any other researcher and essentially without fault. This especially means avoiding the impression of causality when discussing results when association is all that's present. (this is, of course, easier said than done, especially when the statistician is not the PI and does not have the authority to control all the text of a paper) Sure, my colleagues may question why I chose to categorize a variable a certain way or why I'm using a significance level of 0.1 rather than 0.05, but these are matters of professional opinion. If I find a difference in the average length of hospital stay for black patients versus white patients after comparable illnesses, my goal is for that conclusion to be uncontestable. Certainly, one could suggest controlling for other variables or using a larger dataset, but as far as my specific data shows, that relationship is The Truth. And this is where reporting statistics gets so frustrating. Of course we want to generalize our findings and apply them to other problems. But the only things we can really know about are the things about which we have tangible data. Anything, and I mean anything, beyond that is guesswork. It may be educated guesswork, but guesswork nonetheless. And hence the infuriating hems and haws you get from scientists and the sense the general public often gets that we're never in agreement and we're just making shit up. In reality we're (most of the time, I hope) just trying to be true to our ethics.

7 Comments:

I have a pet peeve that college curriculums consider Calculus, and not Statistics, to be the hallmark math course for a well-rounded undergraduate education. We should be teaching students to think critically and understand why and when statistics are valid.

I agree (though of course I'm biased). Then again, having come from an engineering school, it's hard to get away from just how necessary and applicable calc is to pretty much every other science. Not that stats isn't, but both should really be the hallmark of a well-rounded education.

In another as-yet-to-be-developed post about my fantasy of 'fixing' the way math is taught I would ramble on about how to strike a balance in intro stats courses between knowing enough to be curious and critical and the horrible combination of both too much and too little knowledge resulting in bad analyses carried out by people who only think they know what they're doing.

*ahem* While calculus (really, differential equations) is critical to all engineering sciences, I'd like to point out that statistics is also a required class for any accredited engineering program. While we don't hit it nearly as in-depth as a statistician, who really does? Plus, a lot of us take more as graduate students specifically to be critical and objective in our research.

So, I should probably think out my response more before posting, but I'm in the spirit (and at the webpage) now, so I might as well go with it...

1. Every action, comment, act of research, value-judgement will have bias in it. On some level, that's just how the brain works... I can go into the three primary processing heuristics of impressions, but that's a lot of unnecessary psychology/management theory for something that can be summed up as: some bias will always exist, and a professional attempts to minimize it when they're providing an objective opinion.

2. "Cooking" your numbers. This is a common engineering problem, too. On one hand, you're being employed to provide a service, and if your employer wants you to slant a professional decision in their favor... well, it's hard to say no. Yet in the end, *you* are the expert, and it's up to you to assert decisions that are technical in nature. I can design a well for a client that's cheap and shitty if that's what they want, but its my professional responsibility to assert that a different design is a better well because, well, I'm the expert and they're hiring me for that expertise (and that contrary opinion, even if its not what they'd like to hear).

3. A set of professional ethical guidelines are wonderful. The first day of every ChemE class was always engineering ethics. I can still recite the Engineer's fundament canon to this day:

"Engineers, in the fulfillment of their professional duties, shall:

1. Hold paramount the safety, health, and welfare of the public.2. Perform services only in areas of their competence.3. Issue public statements only in an objective and truthful manner.4. Act for each employer or client as faithful agents or trustees.5. Avoid deceptive acts.6. Conduct themselves honorably, responsibly, ethically, and lawfully so as to enhance the honor, reputation, and usefulness of the profession. "

Hmmm... speaking in absolutes like this makes me sound preachy rather than someone engaging discourse. I guess I'm bringing up my anecdotal evidence because I think the current system forces a certain amount of critical ethical system already, and that I, personally, feel no hesitation to do what I feel is right, even when it's not profitable to do so.

I'm glad statistics is not a standard course (for those curricula for which it is not). Bruce Schneier has described the majority of his current career as paying pennance for having written the book "Applied Cryptography" --- which empowered millions of unqualified programmers to write security critical code, leading to uncountably many security problems. That is, we should not empower people to do tricky and critical jobs with a "Dummies" book or an undergrad class. I'm afraid a standard undergrad stat class might just empower individuals to produce lots of bad statistics.

If you really are just proposing a "how to think critically about science and the media" class, then I also disagree: it should be a required class in high school.

PC - I agree, a critical-thinking-type course may be more applicable in a high school setting. In fact, I'd like to see more stats in high school (but maybe they're there now? I know there's an AP stats course now that didn't exist when I was in high school...). I know I hit college without much of an understanding of what was even meant by statistics, in a general sense or as a course or as a major.

As for empowering people to do bad stats - that's exactly the problem I was alluding to earlier. Intro stats courses, in my mind, aim to accomplish two broad goals: 1) to enable researchers to think critically about and design their own studies and know when they need a statistician and be able to collaborate with a statistician and 2) be able to read critically and understand other research. How best to go about accomplishing these goals while reigning people in just before they think they know more than they do...well, that's the hard part.

Sid - as for the stats requirement, you're right, especially at case, practically everyone had to take some level of stats. But I think, for me (and this may just be me playing the victim) it still felt (feels?) a lot like stats didn't get the same *respect* as calc. Every student may have grumbled about how hard diff eq was, but it seemed that they all readily accepted its necessity. Meanwhile, it felt like stats was much more an uphill battle of convincing students that it would be useful and pertinent to their work. It would be nice to see it accepted into the canon a bit more.

Well, Megan, part of that was just the fact that having a major that doesn't end in "engineering" meant you were a SCC* at CWRU.

Mostly, though, I think it was because of what P.C. brings up... you get enough statistics as an engineer to be able to understand terms, but not use it functionally at the level of complexity we need without taking a lot of additional courses. It was sorta like my process control class; a bitch to get through, technically complex, but so idealized as to be close to completely worthless in any realistic setting. If something doesnt seem worth the effort/time, lazy college students will inevitably grumble. Most engineers I know would have actually liked *more* statistics than we got, but without the flexibility to pursue it aggressively, we'd rather not do it at all.