Wednesday, May 9, 2012

Judgment Day I:The politics of beer rating

I've been working on my beer tasting and evaluation skills a lot lately. You see part of it here on the blog in that I've started to publish tasting notes for my own brews. I've also been doing a lot of things like listening to podcast and reading about beer judging methods, and trying to incorporate these ideas into my own evaluation process. It's been a highly rewarding experience, where I found myself suddenly discovering new smells and tastes in beer that I have not noticed before. There's nothing like the first time that you smell a beer differently and you go "By Gad! This doesn't just smell of hopes, it smells like Mango! There's no mango in it, but I smell it!" (It's a little hard to engineer such a moment, but if you can achieve it in the presence of someone else, the startled looks are worth the effort).

In this quest to expand and train my palate I've been tasting a fair amount of Israeli beers. I have two reasons for this:
1. I live in Israel, so I never know how fresh is the imported beer I drink. Local beer, on the other hand, is fairly consistent.
2. There's a fairly small number of Israeli beer raters on, so I get to know the way they rate fairly well. It also means that a typical Israeli beer doesn't have nearly as many ratings as an international brand, which lets me read a few reviews by people who's spectrum of ratings I know, and helps me calibrate my palate against a fairly consistent sample.

I also taste a lot of beer made by other home brewers, and try to have as many people taste my beer and give me feedback. It is, in my opinion, a crucial part of becoming a better brewer.

But lately I've been noticing something: Israeli raters rate Israeli beers higher, on average, than import beers.

It's a well known problem: A guy hands you a beer that he made. He's very excited about it. He thinks it's great. He looks at you with big dewy eyes as you take your sip, excited to hear your praise. But the beer sucks. Or maybe it doesn't suck, but it's just not that good. Maybe you detect the hot-fermentation byproducts, the mash tannins, or just the recipe flaws, doesn't matter. The point is that he's your friend and you don't want to offend him. So you tone it down. You don't tell him that his beer sucks (and by gad, I've had homebrew beer that actually made me want to say "yuck") You say it's "interesting" and "complex" and that you can really taste that special ingredient that he put in. He worked so hard on this thing, and it's his own hand-crafted creation. So you don't want to crush his spirit, you want to let him down easy.

And I think that's a problem. Yes, if someone spat out my beer and said "yuck" I'd be offended. But at the same time, if someone spent five minutes telling me how great it is I'd call him a liar (maybe not to his face, but I wouldn't believe him.) The greatest thing you can do for me, as a brewer, is to tell me what you taste in my beer. How it feels in your nose, and mouth, and throat. THAT'S what I want to know: your honest impression. One of the biggest compliments I've ever got for my beer was from a woman who sipped my Pilsner and exclaimed "I taste saffron!" I don't know where the hell she got that, I certainly didn't put any saffron in the beer, but the fact she tasted it meant that she was actually taking the time to really taste and evaluate my beer critically. And I really appreciated that.

The big problem with this kind of false feedback is that people believe it. And if you keep getting this kind of feedback consistently (and you probably will, because you keep asking the same people - your friends) It may lead you to believe that your beer is, and has always been, great. And that's bad, because it can mean that you never strive to improve. What's the point in trying to tweak your process, in improving your sanitation, in refining your control points,and in controlling your fermentation if your beer is already "perfect"? Why learn, invest, research, test, evaluate, and respond if you "don't need to change anything"? It's a trap. At best, if you buy into these kind of accolades they will lead you to being a mediocre brewer. At worse, it can lead you to think that you are so good that you can go pro.

I know at least one brewery like this: It started with a homebrewer who caught the bug and started brewing in his back yard. Within a year, speared on by a chorus of adoring fans, he opened a brewery. He put in a lot of time and effort into professional equipment, advertising, innovative business plan, and a dozen other things. There's just one problem: The beer isn't good.

Don't get me wrong, the beer is not bad for homebrew, and it has potential. But it's not good enough for a commercial product. The guy spends a lot of effort trying to be a "cool and innovative" brewery, which means that he does a lot of gimmicky beers (something I will talk more about in part II of the judgment series) and is involved in the brewing community in his area. The constant stream of "new, innovative beers" that comes out of the brewery keeps up customer interest and drums up business. But if you ask any of the local drinkers in his area who know how to taste beer, and aren't fooled by dumping a massive amount of hops into a mediocre beer, they'll tell you "he's great, he's a good guy, he does everything right, except for the beer..."

And the sad part about that, and I think the dangerous part, is that when you have a bunch of friends and casual drinkers singing hosannas to your brew, you tend not to accept the few who do try to give you honest advice. I have a friend. A brewer of great beers, greatly educated in the theory and practice of brewing, and (unfortunately) an arrogant bastard. (I think anyone who knows who I'm talking about will agree with that statement. In fact, I think he himself would). There's no way around it: This guy is convinced that he knows the only way to make beer, and that anyone who doesn't do it his way is an idiot. But for all his personality shortcomings, he has a great palate, and he knows how to taste beer. This is the kind of person I want feedback from.

My friend is a bit of a pariah in some parts of the local homebrewing community because of his abrasive nature. But I ask you: If you truly want to improve as a brewer. If you truly want to make great beer. If you truly want feedback on your brew. Who would you rather ask, the guy who's trying to be your friend, or the guy who'll give you his honest opinion and doesn't give a crap if you never talk to him again?

We all have egos. And when you craft something as personal as a home brew. When you put in all the time and effort and do it "the best way you know how", it sucks to have someone take one sip of this thing that you worked so hard on and tell you what's wrong with it. But if we truly want to improve we must put our egos aside and allow our beers to be tasted critically. By ourselves, and by others. Otherwise, we will never be more than the sum of our accolades.


  1. You say Israeli raters tend to over-rate Israeli beers, but could it be that some people just have different taste than yours?
    I always try to be objective about the beers I rate, Israeli or not. Some brewers who I consider friends got bad ratings from me in the past, but when I like a certain beer I shouldn't give it a shitty rating just because the brewer's a friend of mine, right?
    It's great to be critical, it's not so great to be like this film reviewer from some Simpsons episode who just keeps yelling "IT STINKS! IT STINKS! IT STINKS!"

    1. It is absolutely possible that some people have different tastes than I do. I will even argue further that when a group of people tastes together, they tend to calibrate their palates similarly. So if a group of people taste the same beer together, and I taste the same beer alone and under different circumstances, it is possible that I would arrive at a different result than the consensus.

      At the same time, the brewing community in this country is small, and the rating community is miniscule. Everyone knows everyone else, and often the raters know the brewers (I recall at least one instance where of the four ratings for a particular beer, three came out of a tasting at the brewer's house) I'm not saying that's bad necessarily, and I certainly believe that you are can, and have, given bad reviews to people you know. But I disagree with your (implied) premise: You should give a beer a good rating because it is a good beer. Because it's done well, and little or no flaws, and is loyal to the style. Not because you like it. Your personal tastes should, in my very humble opinion, play as little part in the rating process as possible. And that includes your opinion of the brewer.

      Of course, there's a deeper question here: Why do we rate? Do we rate beers because we like them or because we think they're good? I did that once. I rated a commercial beer quite high because I liked it. But you know what? It was not a good beer. And when I tasted it again later, with a more developed palate I was able to say "yes, I still like this, but it does not, objectively speaking, deserve the rating I gave it."

    2. Check out RateBeer Guidelines - it's supposed to be a hedonistic rating, not a "rating to style". We are beer raters, not beer judges. If I like a certain beer - that's what matters.

  2. I think there's also a problem with people tasting your beer (such as, ahem, yours truly) who don't really know what home brew is supposed to taste like. To me the flavor of homebrewed beer (well, yours, but that's all I know) usually seems way too strong, but then I'm the person who would rather order Smirnoff Ice. So would my honest opinion actually be useful?

    1. Actually, yes. If you can get past the initial impression, think about what you are smelling and tasting, and share that, than your opinion would be very useful. If you smell a beer and the first memory you get is of a horse paddock, and you share that with me, it tells me a lot. If you smell it and it smells like my father (who loves cooked cabbage) well then that tells me something too...