If a high-quality site like Metafilter can be crushed by Google, what hope do other sites have?

For long-time web users, the announcement this week from Metafilter — a pioneering online community that has been around since 1999 — was a little like hearing that an old friend is in hospital with a fatal illness: founder Matt Haughey said that due to a dramatic decline in traffic and related advertising revenue over the past year and a half, the site has had to lay off several of its moderators and is essentially on financial life-support. And the name at the center of this drama likely won’t come as much of a surprise: Google.

In a post he wrote on Medium (and in a somewhat shorter update on Metafilter itself), Haughey describes how traffic to the site suddenly cratered in mid-2012, with visits falling by more than 40 percent. At around the same time, Google was rolling out an update to its indexing algorithm — an update known as Panda — which was designed to promote high-quality content and down-grade spam sites, as well as those using a variety of search-engine optimization or SEO tricks.

Not surprisingly, this update hit some “content farm” producers such as Demand Media fairly hard. But Metafilter is the polar opposite of that kind of site: its discussion threads — especially the ones on its Q & A forums at Ask Metafilter — are almost always excellent, with thoughtful responses and high-level conversation. Until recently, the site employed eight full-time moderators to keep the site clean, far more than many much larger sites use.

“The money situation changed one day in November 2012, when I saw a drastic reduction in traffic and revenue to Ask MetaFilter. I double-checked to make sure the initial estimates were correct, and it appeared that Ask MetaFilter lost 40% of its traffic overnight.”

The cause of the traffic decline is a mystery

As with most things involving Google and its algorithms, what happened to Metafilter is almost impossible to diagnose, because the search giant’s methods — and the motivation for any changes — are a black box. Haughey theorizes that the change occurred around the Panda update and was exacerbated by subsequent updates, but Search Engine Land founder and Google expert Danny Sullivan pointed out that the dramatic drop-off in traffic doesn’t really line up with any of the company’s major algorithm tweaks.

If you’re interested, Sullivan has since written a much longer description of what Google does to sites via its various algorithm updates and “filters” like Panda or Top Heavy, and he agrees that Google could do a much better job of explaining to sites like Metafilter what they might have done and why, and how they might be able to alter their ranking or indexing to make it better.

Although it’s difficult to quantify how much Metafilter has fallen in Google’s search results, David Auerbach at Slate magazine did an informal survey of some results that should arguably have included pages from the site, given the amount of content and links related to that topic or search phrase, and found that in a significant number of cases Metafilter ranked a lot lower than it did on Bing: In one case the result was #2 on Bing and #60 on Google.

Google’s methods remain a black box

At one point, Haughey says that he tried to get Google to talk about what the site was doing wrong, or suggest steps that he could take to ensure that Metafilter’s content was ranked higher, but all he got was suggestions from AdSense executives that he should buy more advertising — although Sullivan notes that Google’s top spam cop, Matt Cutts, said on Twitter Wednesday that he has discussed the issue with Haughey at some length.

Whether Cutts is able to help or not, Metafilter’s experience sums up one of the main concerns that many publishers have about Google’s dominance in both search and search-related advertising. If having high-quality content like Metafilter isn’t enough to satisfy the search giant, then what hope do other sites or publishers have of catering to the company’s wishes?

It’s worth noting that Haughey points out a few other macro factors that could be affecting his site’s performance, including the decline of advertising itself: In other words, the theory that the web has reached “Peak Ads,” and traditional web strategies — including Google’s AdSense and other methods — are just not as effective as they used to be, and likely never will be again. As Haughey notes in his Medium post, this state of affairs is arguably exacerbated by the shift to mobile, where traditional web advertising also doesn’t work as well.

Shouldn’t quality matter?

One factor Haughey doesn’t really dwell on is that many online communities have a natural life-span, and it’s not uncommon for them to gradually decline in popularity — it happened to Slashdot (whose founder Rob “Cmdr Taco” Malda now works for a content-recommendation startup called Trove) and to Digg, and to plenty of other pioneers as well. That said, however, a gradual decline in popularity doesn’t really jibe with the sudden 40-percent drop Metafilter saw.

In the grand scheme of things, the fact that a small online community run by a handful of people is in financial trouble isn’t an earth-shattering development — Haughey has always run Metafilter as a kind of “lifestyle business,” without any of the usual Silicon Valley pretensions of grandeur or world-changing billion-dollar IPOs, and so even if Metafilter ceased to exist it probably wouldn’t be noticed outside a small group of old-time web users.

But at the same time, when a site that is so obviously concerned with quality — and so clearly driven by the pursuit of thoughtful content and interaction — gets penalized to such an extent, it makes you wonder what else is happening inside the confines of Google’s black box. Unfortunately, we will probably never know.