Friday, July 22, 2011

The Fallacy of Ratings

This is a wine review - and it is a rant. It is also well-trodden territory: Wine writers, producers, and consumers alike decry the unfairness of wine scores, unless, of course, they lean their way. Well, boo hoo.

Here's my rub: through repeated panel tastings, wines awarded prestige through ratings fail to impress with any consistency or correlation to their scores - especially at the higher ratings. This is across the board as far as publications go, too: Parker, Spectator, Enthusiast...it doesn't matter.

Okay, so it's further evidence that wine is as subjective as art. Different strokes for different folks. No big shocker. But consumers aren't being done any favors, especially at the highest end. Case in point is this wine which was (probably mistakenly) sent as a sample - the 2008 Two Hands Shiraz Bella's Garden from Barossa Valley.

Here are the combined tasting notes from three evaluators:

Near black-crimson, intimidating looking. Has the viscosity of blood. Serious just sitting in the glass. Clocking in at a whopping 15.8%, the heat on the nose moves spices, oak, and black currant notes quickly. Complex palate. Rather than being fruit bombastic, this racy Shiraz shows its Viognier proudly. Immediately enjoyable, but possibly an ager. The $60 is tough to swallow unless you are an Aussie aficionado/collector. In the final analysis, though, there are many Australian wines that deliver equivalent pleasure for a quarter the price.

The tasters knew of the price tag and the wine's origin, but nothing of the brand or rating. What they didn't know (neither did I) is that this wine was given 94 points and ranked #2 on Wine Spectator's Top 100 Wines of 2010. There's no question that this is a pleasant wine to drink, but really? Number 2?

This is far from an isolated incident, too. Parker called the 2008 Chateauneuf du Pape from Domaine de la Janasse, "...outstanding...one of the stars of the vintage." It was okay, but nothing to right home about. Schild's Shiraz was also given 94 points - we found it to be rather pedestrian. This list goes on and on.

So, what's my point? Nothing new. Just that people should let their own palates guide them to their definition of what's good and what isn't.

6 comments:

On a more serious note, had a Purple Haze Syrah/Viognier from E11even Wines, a 2nd label from Andrew Murray. Basically identical in composition and alcohol as this wine, only more enjoyable and a fraction of the price. Wish there was more of it available!

"Schild's Shiraz was also given 94 points - we found it to be rather pedestrian." Steve, I couldnt agree more. I was expecting this Shiraz to move mountains considering the praise for WS and others...but it was rather just OK. At least it is a relative bargain. I bought a 6 pack on pre-release to see if this wine could live up to the hype. It seemed what Australia needed to help get out of its recent funk...but, the wine was just flat to me any my wine friends. Note: to be fair, I may have received a "secondary" bottling and not the actual wine that WS tasted. Not sure if you heard about what the Schild winery did after it got the great score??? They got some bulk wine and labeled it with the same exact label. Sounds criminal to me. Anyway, interesting post. Bottom line...it is hard for wines to live up to the hype of a great review. Sometimes the anticipation is sweeter than the actual taste...true well beyond the realm of wine...dont you think?

PS...glad you enjoyed the Purple Haze...still not exactly released...lots more yummy and affordable wines to come from the e11even wines project...stay tuned.

Thanks for chiming in, Andrew. Yes, fully aware of the Schild fracas. I had it pretty early on - not sure if it was before or after WS showered it with jubilant praise, but the "second release" was supposedly only released to the Australian market. Now, putting aside the polite terms, let's call it what it is: fraud. While technically it might be legal, there is little difference between what they did and forging a work of art. No doubt the market will deliver Schild its comeuppance in future vintages. Heck, it already is: you can find plenty of that Shiraz on the shelf right now. Regrettably, though, that kind of spoil is contagious, as we've seen in Montalcino and Burgundy.

So what is the average consumer supposed to use as a guide in buying wine, especially at higher prices, if not scores? I look at ratings as a measure of risk. And if I want to get something really nice, what am I supposed to get if I can't use ratings to point me in the right direction?

Great points, Lisa. Alas, there are no simple answers. Whenever you invest in something, you want to do as much due diligence so as to maximize your reward over risk. Scores are the natural tool to gravitate towards in doing that due diligence with wine. But as this piece argues, that approach is as fraught with peril as evaluating a stock based on last quarter's earnings.

So, there are other avenues available to you, like this blog (and a thousand others) and community forums like Cellartracker.com where regular Joes weigh in on wines. Ultimately, though, it helps to find a trusted source for advice like what you seek. Not trusted in terms of being reliable or impartial or beyond reproach, but in terms of sharing your particular values. And that source might be a retailer, a wino friend, or even a blogger.