11 Oct 2011 @ 4:16 PM 

One of the most-hit columns I’ve ever written on Top Fermented was a “Beer Advocate vs. Rate Beer” column. It raises ire. Some people like the fact that I attempted to (poorly) apply statistics to compare the ratings on each site. Other people have bitched and moaned about how it’s a steaming pile of turd, which I won’t necessarily argue with – it has flaws. I’m pretty sure I even say that in the article itself.

Anyway, I see all of these links point here and I read them all and contemplate them and never really say anything because.. hey.. it’s like a two and a half year old column at this point, and it’s kind of meaningless now. Ratings systems are continually updated and people continue to use the site in new and different ways. However, the rating sites still over-appreciate dark, high alcohol, and hard-to-find ales and well-made low alcohol lagers consistently under-perform.

I’ve thought about it a lot and I am now of the opinion that wholesale beer rating is really to a point that it is no longer useful and, in fact, might even be detrimental to the market as a whole – and I don’t just mean Beer Advocate or Rate Beer, but beer review blogs, etc., and anon. The noise-to-signal ratio is just out of whack and the results are being given gravity that they don’t deserve.

Before the flames and trolls show up, let me state my case:

People rate beer by measures that are too subjective

Plain and simple. By and large, people rate beer based on whether or not they liked it, not whether or not it’s a good beer. Believe it or not those are two different things. I can’t stand Bud Light, but I won’t tell you that it’s a poorly made beer. It’s excellently made beer if you want a lite (yeah, I spelled it) lager. But it gets a 0 on Rate Beer (a 1 within the style) and a D- at Beer Advocate even though it is essentially the definition of the “light lager” style. Why? Because it lacks technical brewing skill and is rife with off-flavors? No. Because the bulk of the people who are rating it, like me, hate it. Rather than disconnecting themselves subjectively to actually answer whether or not it’s technically well-made and matches the style, they rate their own taste in graduated values of suck.

I do think that there is value in being able to have a list of ratings of beers that you have enjoyed for your own reference. It’s one of the reasons that I like Untappd – because it gives me a list of my own ratings for me to reference later. I don’t always remember a beer three months later. Have I tried this? Did I like it? 5 stars says, “Yes!” But just because I like it doesn’t mean that somebody else will. Taste is subjective. I like a really wide range of beers, but give me something with a load of Nugget hops and I will always, always hate it. 1 star-only and man did that suck. But! That doesn’t mean you won’t like it, so why should my personal rating mean anything to you?

If it’s a useless measure, we shouldn’t be using it to judge beers with.

There’s no way to tell that people are tasting good beer

And by good I mean “like the brewer intended”. Not old or oxidized or through infected taplines or in dirty, frosted glassware or drunk by a smoker or someone with an asshole for a mouth. Certainly, some people will note in the comments of their review about how it was served or what it looked like, etc., etc., but one look at the top comment under Bud Light really says it all:

…serving type was shot gunning at the football game.

Indeed, byteme94. I will now take your D+ more seriously because I know you put a lot of thought into it for those 4 seconds while it was passing through your esophagus. Was the fact that you didn’t immediately throw up what saved it from a D- or an F?

If someone is tasting a beer out of a dirty tapline and (and this is important) they don’t know what a dirty tapline tastes like, they think they just have a shitty beer and there’s no way for me, as a reader, to tell if this is in perfect serving conditions or if this is someone drinking beer out of their cat’s old food dishes before they give a beer a score (“drunk from a straight-sided shallow goblet”, indeed). I’m not going to look through 3,000 reviews. I’m going to look at the aggregate score. If the aggregate score is a composite of unreliable measures, then the aggregate score is unreliable.

There’s no way to tell if the people are good at tasting

Let’s take, for instance, Geary’s IPA in which the first review – which gives it a B (which is decent, if you consider C to be average) – mentions the word “buttery” twice. Once in the aroma and once in the flavor. He didn’t really care for the butteriness of the malt. Of course, he mentioned that he wouldn’t really expect bitterness or alcohol in an IPA, either. Now, I happen to know that Geary’s is brewed at Shipyard, and that Shipyard’s house yeast is Ringwood which has a VERY high flocculation rate. It tends to drop out of the solution really early and doesn’t really remove diacetyl (which tastes like butter) from the beer like it should unless you do some awesome tricks to keep that yeast in suspension – which Shipyard is generally pretty good at.

IPA shouldn’t be buttery. Malt does not taste buttery. This is an off-flavor. But the reviewer doesn’t know this (or that an IPA should be bitter, sadly). He just thinks (correctly) that it tastes like butter, and while he doesn’t really like it he also doesn’t know that it’s not supposed to be there at all so he doesn’t judge it as harshly as he could and maybe should. Or to look at it backwards, he is judging it as though the butteriness is supposed to be there, because he doesn’t know that it isn’t.

Is this a good, honest review of this beer? It certainly reflects whether or not the drinker likes it, but does it reflect the quality of the beer? ie – Why should this B count with the same weight as someone’s C who does know that their beer is diacetyl heavy? How do I know if the person who is reviewing the beer knows enough about the beer to give a good review? Just because you drink a lot doesn’t make you an expert. It just makes you drunk.

(I am positive that at this point in the article, at least one thread will start on a forum somewhere to discuss whether or not it matters if a beer is well-made if you like drinking it, anyway. Related: Who cares who makes your beer if you like drinking it? Answer: I do.)

The internet is untrustworthy in general

Sorry kids, but I just don’t have any reason to trust you. Just because a lot of people rate something doesn’t mean that there’s any sort of reasonable quality involved. You know that saying that’s something like, “50,000 people can’t all be wrong”? Well – actually, they can. It happens all the time.

A significant portion of this country believes that science and math are just these things that the educated elite make up to try to perpetuate grant funding because paying yourself off of grants is sooooo awesome. They believe things like vaccines are bad but polio is kinda okay. They believe that man and dinosaurs used to co-exist. Why on earth should I trust you, the internet, to know enough about beer to give me a decent recommendation if you can’t get broad “society has moved on” issues correct?

In Summary

Fact: You can’t measure something with an unreliable tool. If I’m allowed to make my own ruler that just has however many inches I want on it at whatever random intervals, I can use it to build the same thing every time. But as soon as I give you my plans you are up a creek without.. well.. a ruler. Have fun defining that cubit, bucko, because I measured it using MY forearm, not yours.

There’s no good way to cut through the noise of beer reviews to find out which ones are worth paying attention to and which ones aren’t. Since there’s no way to calibrate the tasters to make sure that they’re all tasting with the same objectivity, then there’s no way to say that any given set of ratings is even reasonably reliable and I won’t waste my time with them. Until we have some sort of Cicerone-weighted rating system or something like that, I’m calling shenanigans on beer rating, especially wholesale ratings sites like BA and RB. Their data is no longer worthy of consideration, by my estimation.

Make your own ratings and decide what you like for yourself. It’s far more valuable in the long run.

These ratings are being put forth as guides for consumers

Let me quote something to you from the comments of a blog that I ran across that I’m pretty sure sums up common sentiment. I know that I should quote who it’s from, but I don’t know them personally and I don’t want to get into any sort of pissing contest. This quote is in reference to a post recommending shelf tags from Rate Beer and Beer Advocate in retail establishments, much like you would see shelf tags from, say, Wine Spectator.

I do appreciate that the rankings are from a consortium of dedicated drinkers compared to wine, which historically was dominated by one individual or several publications.

Indeed. You know what I hate? Being able to make informed decisions based on reliable, consistent data. What I prefer is to make random guesses based on completely unreliable anonymous data. I mean – who needs Consumer Reports and a trained panel of experts when I can get a product rating from BoobLvr67?

That is the equivalent of trusting anonymous online ratings for beer (or anything, really, but let’s stick on topic).

What I’d Like To See…

…is some sort of rating system from people who are actually known trained tasters – Cicerones and/or BJCP judges – with ratings ranked in importance based on how skilled they’ve shown themselves to be. That would be better information. There’s still individual taster differences, but at least those tasters have been moderately calibrated. At least there’s a starting point beyond, “I signed up for the website.”

That’s a rating site I’ll trust, and those are shelf tags I want to see in retail establishments. Until we can get there, I’m dispensing with wholesale beer ratings in general.

Share
Tags Tags: , , , ,
Categories: appreciation, blog, industry, op-ed
Posted By: erik
Last Edit: 11 Oct 2011 @ 05 00 PM

EmailPermalinkComments (54)
 01 Jul 2011 @ 10:12 AM 

If you stopped by the blog over the past couple of days you might have noticed a few strange things afoot. For a while, we were down entirely and once we came back up, you weren’t looking at TopFermented at all, but an old beer blog from 4 or 5 years ago called “The Brew Guide” which had a short, disjointed life.

But, we’re back and better than ever on a faster server. Will you notice anything? Probably not, but I sure as hell will. So now that we’re back up and running (and I get to replace my fried computer this weekend), expect some more content coming through in the next couple of weeks. Seabass and I will be shooting another video session in which I hear we’ll probably be talking about packaging and I have a few visits to NC Breweries coming up. In fact, I have a LOT of those coming up all the way through July and August.

Oh, and hopefully we’ll have some Mystery Brewing news to share within the next week or so, too.

Until then, welcome back for the first time to the place we’ve always been.

À votre santé,
Erik

Share
Tags Tags:
Categories: blog, meta
Posted By: erik
Last Edit: 04 Jul 2011 @ 02 25 PM

EmailPermalinkComments (0)
 25 Jan 2011 @ 3:14 PM 

And before you ask me to never use the word “sluice” again, here’s a lovely picture of a sluice from Wikimedia Commons:

I would also like to relay that “sluice” is a surprising safe Google Image search.

We will now carry on with our regularly scheduled blog post.

So, what’s coming down the sluices!?

I’ve been conspicuously silent across both this blog and Mystery’s blog (where this, incidentally, is being cross-posted, if you’re reading this at Mystery’s blog, you may want to check out Top Fermented), for the past couple of weeks and that’s primarily because my days have been turned into a twisting mass of odd jobs, manual labor, staring at the wall waiting for inspiration, and alternately burying myself so deep into work that I forget to eat. A good chunk of this has been keeping me away from writing.

But it hasn’t been keeping me away from the computer. More on that in a sec.

I’m on a more regular schedule now, where I’m actually spending 3 days a week “at the office” so you should be seeing a few more blog posts popping up here and there.

Also popping up should be the fruits of (some of) my labor, so here’s a little preview of what to expect in the next couple of weeks:

Educational Opportunities

In case you haven’t heard, myself and a couple of excellent friends organized and hold a monthly beer Meetup here in the Triangle in NC called Taste Your Beer for lack of a better, more inspiring, name. It’s been received pretty well and people seem genuinely excited to learn more about beer – not how to make it, but how to enjoy it, and just more about beer in general. So when I heard that there were upcoming Cicerone exams coming to Raleigh, I had the idea to make a study group for it.

However, after thinking about it, I thought – why limit this to just people who want to become Cicerones? Lots of people want to learn about beer but don’t necessarily have the desire (or the work experience and wallet) to become Cicerones. That’s why, starting in February, I’ll be offering beer education classes at my location at Mystery Brewing. It’ll be an 8 week class meeting once a week (with a few exceptions) covering beer from ingredient cultivation to serving and food pairing including off-flavors and style samples. It will cover the Cicerone exam content thoroughly so if you, like me, want to take the Cicerone exam in April or June, then this should act as an excellent study guide. However, if you just want to learn about beer then that’s cool, too.

Look for more information about these classes popping up in the next few days. We need to get going soon to be ready for the Cicerone exam AND the World Beer Festival.

New Website

With a new brewing company comes a new website. The blog over at mysterybrewingco.com will soon be going away for a more robust website with some features that I think will be fairly interesting to people. Among them are the normal kind of website things: discussion boards, a news feed, info about the brewery, social media and that sort of crap. But here’s a little preview of some of the other things I’m working on (not all of which will be up and running immediately):

  • A check-in point/badge system specifically for Mystery Brewing. Think FourSquare, or Untappd except you actually have the chance to get REAL REWARDS if you earn the right badges: Discounts on brewery merch, beer, private brewery tours, beer, t-shirts, beer, stickers, and probably, at some point, beer. This should launch with the new website, even if beer won’t.
  • A Mystery beer genealogy tree. I am quite proud of the fact that all of my beers started as homebrew recipes, and I am telling you now that they are all going to evolve over time. Recipes I have now may spawn other recipes in the future. This beer genealogy tree will be a way to find out how all Mystery beers are linked together, batch to batch over time. It will serve as a means as helping people find out both what they enjoyed about a beer and what new beers they might enjoy. Once all the equipment drops into place and Mystery beer starts hitting bars and restaurants, this will also serve as a way to track which batches of which beer are out in the public and where you can find them.
  • An ongoing art contest. I am a big fan of the arts in general. I went to an art school for my undergraduate experience and was, shall we say, intimate with the art school, even though I was only a performing artist, myself. I would like to take the opportunity to showcase art through Mystery. In specific, I will be announcing an ongoing art contest of sorts through which artists of any sort – professional, amateur, painters, web comics, whatever – can submit artwork for use to represent beer in our repertoire. The artist who’s work is chosen will receive money in return for the use of the art, as well as a royalty for every piece of (non-packaging) merchandise sold using the artwork. (Since we won’t be in bottles for a good long time, we’re talking posters, t-shirts, etc.) More details on this later in the spring, but artists, start thinking Evangeline.
  • Weekly updates on progress in the brewery. Things are starting to pick up speed and while anybody who is part of the classes up above will be able to see things starting to pop up around them, a lot of people don’t know what exactly is going on in there, so we’re going to get into some detailed updates on how we’re progressing toward getting beer on the market, even if that update is why progress isn’t being made. Back when I started Top Fermented, this is one of the things I really wanted to do is get into the nitty-gritty of what goes on behind the scenes when a brewery is opening. For the most, especially when it’s come to financing, I’ve felt like it was either a little boring or getting into detail would get into confidentiality issues with my partners. Now that we’re moving past getting money and into (*whimper*) spending it, I feel a little more like I can let people behind the curtain. Prepare yourselves to see week after week after week of.. ermm.. well… pictures of an empty cement box. Yaaay!
  • More from me about the industry in general. I’ll be folding Top Fermented into the new website. It’ll still exist on the original domain and function independently, but it will also be integrated into the new website as the brewer’s blog. It means no more separation of sites and it should mean a more rigorous update schedule. It might also mean that I piss more people off that I probably want to retain the respect of as I voice my opinions, but.. ermm.. well.. that sucks.

    Okay – this part isn’t nearly as exciting to you as it is to me. Still. I’m excited.

Kickstart-y Goodness

And no, that doesn’t mean that I’m starting another Kickstarter project (yet), but Kickstarter backers will remember that there are still homebrew recipes to go out, Irregulars memberships to revel in, beer dinners to eat, and video chats to watch. I haven’t forgotten, and there will be movement on a couple of these things soon.

And more.. much, much more.

If I’m running into any sort of problem, lately, it’s the fact that I have more ideas for things to do than I have resources and, frankly, spare neurons for processing. The important part that my next blog post should be a snark filled rant about some sort of craft beer segment piece and not one of these lame update sessions.

But! The future is bright and there’s beer there. Join me!

À votre santé,
Erik

Share
 24 Jun 2010 @ 9:38 AM 

No shit, buddy. You start starting a brewery and then you fall off the face of the blog? Way to show dedication. Hell, you showed up for the Session that you hosted and then.. what? Posted a few pictures of a half-done tattoo and fell into obscurity? Good work!

Okay. Touche. I’ve been absent. My bad. In truth, I’ve been working on a few articles and just haven’t had a chance to finish them up (though I did get one out – non-beer related – for Intrepid Media) and.. and.. I haven’t forgotten about you, internet. It’s not you, it’s me. I swear!

In fact, just to get things rolling again before I drop my bomb post about craft market dichotomy, let me give you a couple of posts from OTHER blogs that I’ve found interesting, lately.

Nate at Thank Heaven for Beer wrote a great piece yesterday about HR 4278 and why reduction of excise tax on small brewers is NOT a stimulus package, and why it is, actually, fair. This follows up another great post (with equally great discussion) about beer reviews and whether or not they’re legit if they used terms you can’t recognize.

Seriously – if you like to read about the craft beer industry, go read both of those pieces AND all of the comments. They’re well worth it. Thoughtful pieces, thoughtful commentary.

Ken at The Hop Press wrote a really neat piece about Sierra Nevada’s use of a Gas Chromatographic Mass Spectrometer w/ Olfactory to pick up really specific fruit flavors and aromas from new hop varietals. It’s a really fascinating piece. I enjoy the implications of what it means for the future of flavor in craft beer, which I would like to define as: “More awesome is yet to come.”

And as for me? You just watch and wait. And then take me to the cleaners in the comments on my next post.

Share
Tags Tags: ,
Categories: blog, meta
Posted By: erik
Last Edit: 24 Jun 2010 @ 09 38 AM

EmailPermalinkComments (1)
 05 Jun 2010 @ 5:59 PM 

Thank to everyone who took part in this month’s session. We had a lot of great response. It was great to see such a breadth of posts, and I was happy to see some people contribute who haven’t in a while. Here for your reading pleasure:

Mario at Brewed For Thought defines the “California Session Beer”, made with umph, but not too much.

The Beer Nut takes us on a small tour of two session beers: Hopback Entire Stout and Breconshire Night Beacon.

John/Devoid of Beertaster.ca talks about his favorite lawnmower beers.

Steph Weber, one of the many talented bloggers of the Hop Press posits that session beer is a personal definition, and tells us hers: easy to enjoy while chatting with friends.

Chris at Notch Session Ales gives us a look at his recent article for Beer Advocate magazine in which he says he wants to take the definition for session beer back from the 6% crowd.

Derrick at Bay Area Beer Runner likens session beer “great background music during an evening out with friends.”

Jay at A Beer in Hand (is worth two in the fridge) tells us about returning to his session stout after an affair with IPAs.

Lew Bryson goes to town at The Session Beer Project, puts some old demons to bed, and reiterates his definition a session beer in the face of the bigger burrito. You’ll never think of a ball-washer at the golf course the same way again.

Steve at All Good Beer hopes that craft breweries will continue to expand the session beer market, even though they’re not necessarily headline-grabbing beers.

Brian Yaeger, author of Red, White, and Brew says that if he had a lawn, he’d mow it with Anchor Steam. Well, no. He’d mow it with a riding lawnmower, but he’d drink Anchor Steam. He also waxes eloquent about the low ABV brews at this local hangouts.

Sean at Beer Search Party suggests that the reason that the Session Beer Train has not yet pulled into the station is because of America’s hypocritical self-view of over-consumption, or perhaps that many craft beer drinkers see session beers as “too macro.”

Tom at Lug Wrench Brewing asks you to be a rebel and go drink a session beer.

Someone at Beer Made Clear (there so many of you guys), who apparently has family neighboring mine in Northern Maine (yeah Fort Kent!) and tells a story of discovering Shipyard Fuggles IPA in during a session which he describes as “all-encompassing conversations which start with beers but end with opinions; on politics, culture or the nature of good and evil.”

Jon at The Brew Site gets back to the review and gives us a one of Gone Fishin Mild Ale, from Beer Valley Brewing.

Peter at A Better Beer Blog notes that session drinking is “essentially the guts of the craft beer movement” and hopes that craft brewers will start to “sessionize” everything instead of “imperialize” everything in the future.

Stan Hieronymus at Appellation Beer tells that old joke about the Englishman, the Belgian, and the Czech, notes that session beers are different for everyone and wonders whether the appellation “session” needs definition at all.

Seabass, from Natty Greene’s, posts from inside aussenhaus where he compares craft beer to coffee and wine, and notes that session beers are the greatest delight and the hardest to make, and defines session beer as “deliciously non-intrusive, self-explanatory, [and] a good companion in life.”

Alan at A Good Beer Blog notes that what is probably stopping craft beer from growing is the industry’s inability to take advantage of the session market.

Jim at Two Parts Rye approves.

Jay Brooks of the tells us not of session beers, but of the sessions themselves, and shares the story of the founding of his own regular session, The Philopotes Society.

And finally, my own post where I note that session beer, to me, is one that can be enjoyed without effort and that, yes, please, I would like some more.

À votre santé,
Erik

Share
Tags Tags: , ,
Categories: blog, Sessions
Posted By: erik
Last Edit: 07 Jun 2010 @ 07 09 AM

EmailPermalinkComments (11)
\/ More Options ...
Change Theme...
  • Users » 130221
  • Posts/Pages » 204
  • Comments » 2,674
Change Theme...
  • HopsHops « Default
  • BarleyBarley

About



    No Child Pages.

Shirts



    No Child Pages.

Tour



    No Child Pages.