The problem is, the "Core" fans are too small an audience, so, if you're doing so poorly that you need to bow down to them, you have already lost, it's already too late.

That's fair enough, but it would seem to me the concept of listening to your audience entails a bit more than just blindly accepting whatever the 'Core' fans say they want. It's already been pointed out that there is clearly a difference between listening to your audience and allowing them to control everything you do. And I'll add to that that the 'core' fans don't constitute the entire audience, either, just a part of it.

The "core" audience are the only ones that are going to try to dictate to you how the show should be done. Just about everyone else is either going to watch it and be happy or stop watching, so, you can only assume, that whatever was happening on the show when they bailed, was the wrong thing to do (Or if the audience grew it was the right thing to do).

You'll also have the folks who want to complain about political points, such as people whining about how they're not going to watch the show anymore because of the addition of a Gay character (And most of those complainers don't actually watch the show, they're just Zealots against Gays who pretend to be disgruntled viewers to make it appear viewers are pissed) or other similarly hot political buttons. But the General audience isn't really going to give you much feedback other than watching or not watching