To say "I tried a hat and they don't look good on me" is like saying "I tried suit and they don't look good on me." Hats may be more finicky in appearance since they are on top of your head and frame the face, but I would find it hard to believe that a person simply doesn't look good in any hat. Like shoes, suit, tie, etc. would it not be a matter of finding the right hat? Color, style, fit, texture, brim, curl, crown, bash, ribbon, finish, etc. all come into play.

This isn't to imply that one has to wear a hat or even has to like or want one. But like a suit, chances are good there is one, or likely several out there that look good on you.

I wear a fedora in bad weather in winter, but it's had the daylights beat out of it, so at least it doesn't look I just brought it out for a costume party. My girlfriend hates it.

I tend to think fedoras are less conspicuous in a cold drizzle, say, than on a balmy spring day. For guys under 50 or so, they need the right setting, and they need a certain attire, either very dressy or a little eccentric. (My stuff leans toward the latter category.) The hat and T-shirt combo so popular with fat guys ought to be eradicated.

In summer, I wear a panama, which my girlfriend considers equally crazy. But I think she finds it more appealing than the beat-up fedora.

+100000000000000000000*infinity

I approve of all of this. Try an akubra stylemaster for a slightly more casual hat.