Beer Thief Homebrew Competition (Newfoundland)

[mcgster]

That is crazy. Is that your IPA that got a gold lately, and silver in the BOS round? and someone gave it 10 points? Was that the same person who gave the stout 10 points? That might be as high as he goes, and you actually got 10/10 !!!

I think, or at least Chris (Chalmers) said, in BJCP sanctioned competitions, the scores have to be within a range of each other, and that's why you often see scores crossed out and changed on score sheets, if there is a wide variance, judges will discuss and adjust their scores to bring them closer together.

Yup, its the very same batch and it would have been judged around the same time lol.

The same two judges gave the IPA and the STOUT the ultra low scores.

[mcgster]

I can't comment on this specific comp, as I know less about than the entrants, but that is a large variance.

From http://www.bjcp.org/judgeprocman.php" onclick="window.open(this.href);return false;:
Once all of the judges at the table have finished filling out their scoresheets, they should discuss the entry and their scoring. Keep in mind that some people are more sensitive to certain flavor and aroma compounds (such as diacetyl or DMS) than others. Scores should be within seven points, and preferably within five or fewer points. Adjust scoring as necessary, and write the final assigned score on the cover sheet and flight sheet. Final assigned scores are not necessarily an average of the individual scores.

This is what i have always gone by, if my scores are +/- 5 points i tend to think its fine.

The average variance for my beer is around 3 points for most BJCP comps i send to which is really good!

[Tony L]

Mine were all over the map also. I entered the same beer twice with the only exception that I bottle conditioned a few brews before I kegged it. I experienced scores not ever seen by myself ever, by one judge ( to be fair there were no BJCP judges there and they surely did their best, but one outdid all the rest )

You can see some discussion here http://www.beerthief.ca/forum/viewtopic ... 5&start=80" onclick="window.open(this.href);return false;

[groulxsome]

Ah, yes. About the variability! I've posted more about it over on the Beerthief forums, so you can see my full explanations there.

This was not a BJCP sanctioned competition, so we didn't do the number adjusting. There were a couple reasons, but mostly it was because we had no proper sanctioned BJCP judges, we wanted to give each of our judges personal space. From the other thread:

Thanks for the comments! Keep them coming, we want to make the next competition even better.

A couple things on rating statistics. We considered removing the top and bottom judge for each beer, which would bring the ratings much closer together. After trying out the math, it really didn't make much difference to the overall averages, so we scrapped the idea in favor of more information, even if it was less standardized.

Our judges were, obviously, not BJCP trained, so a lot of personality showed through in each scoring. I might as well share some of this info here, since it's all calculated!



That might give a little more information on each judges' tendencies. In general Tom rated beers lower and Raymond rated them higher. Often when we went over our scores after judging (since we didn't talk when judging) we noticed how crazy a range there would be. Someone would get something Oxidized, other's wouldn't. Or someone dinged something for being too English when others thought it was acceptable for the style. We probably could have massaged them after to make them more closely agree, so we'll be looking at doing that next time.

Also, it's hard to tell if that standard deviation is a bearing on a judge's range or the actual range of beers submitted to the competition. After trying some of the leftover beers on my own later, I tended to agree with my comments about the beer, so I was pretty happy about that.

Anyway, hope some of that is useful. Thanks for the feedback! The hardest part of the whole thing was actually giving out a bad rating to beers, since I know how hard people work on them. I had to garner my courage and try really hard to be a lot meaner than I actually am! Glad no one is too upset about their scores.

So, yah, I mean, we tried to give all the judges a little primer on how to use the BJCP-esq scoring we were using, but it didn't totally work. I (Chris) tried to make up for it by writing as much reasoning as I could, though that was harder since some folks don't brew and are more used to wine competitions.

We really tried hard and we're working on improving the system (and our judge training, hopefully BJPC too) for next time. Thanks for the comments everyone!

I felt like things were scored a little harshly at the end of the day. Some very tasty beers didn't quite get scores that I thought were high enough, especially in aggregate. There were some great beers! And some ended up with 32. If you want to know anything else, let me know!

[mcgster]

I think next year you should have less judges, and ensure they can score the beer similarly.

It is a lot of work to put beer in shows and the reason it is worth it is to get feedback from judges... but its hard to even comprehend what this judge was scoring...

NLHBC - 2014 - Category 14B - 02.jpg

NLHBC - 2014 - Category 14B - 03.jpg

[Jimmy]

I think next year you should have less judges, and ensure they can score the beer similarly.

It is a lot of work to put beer in shows and the reason it is worth it is to get feedback from judges... but its hard to even comprehend what this judge was scoring...

NLHBC - 2014 - Category 14B - 02.jpg

NLHBC - 2014 - Category 14B - 03.jpg

[Keith]

I think the first guy got drunk by the time he wrote #14 haha. I can't read it all.

[CorneliusAlphonse]

1/20 for flavour is crazy. I don't know what I would mark so low, maybe water entered in dipa category

[GuingesRock]

1/20 for flavour is crazy. I don't know what I would mark so low, maybe water entered in dipa category

Does that say under qualifications “nano planned in Toronto?”

Also ticked; Acetaldehyde, Oxidized, Boring, and Major flaws, Not to style.

If I had a good beer like that, sent it in, and someone wrote that and sent it to me. I might wonder to myself WTF. Might even say something about it as well.

Looks like the other judge did a good job though, so I’d recommend keeping the valuable score sheets and throwing out the ones that are not useful to you. (I’ve done that before, but don’t tell anyone.)

For this competition, you were required to submit full recipe details. That is normally optional. With that in mind, I was interested to see one of the judges is in the process of opening up a nano and I wondered if that might be a conflict of interest? At least for something like Garrison, Big Spruce, Hammond River etc. they are up front and say they are looking for a beer to brew commercially.

[GuingesRock]

Did you notice that the score of 1/12 for aroma, the number 1 is underlined. That seems aggressive to me, and not in the spirit of things at all.

[GuingesRock]

How did judge #2 do with picking up Blackcurrant in the aroma? Did you have Bramling Cross in there? I'm not trying to be critical, I thought the judge might be right and wondered how you did that. I like blackcurrant.

[GuingesRock]

I think the first guy got drunk by the time he wrote #14 haha. I can't read it all.

Her name is Christina

[mcgster]

1/20 for flavour is crazy. I don't know what I would mark so low, maybe water entered in dipa category

Does that say under qualifications “nano planned in Toronto?”

Also ticked; Acetaldehyde, Oxidized, Boring, and Major flaws, Not to style.

If I had a good beer like that, sent it in, and someone wrote that and sent it to me. I might wonder to myself WTF. Might even say something about it as well.

Looks like the other judge did a good job though, so I’d recommend keeping the valuable score sheets and throwing out the ones that are not useful to you. (I’ve done that before, but don’t tell anyone.)

For this competition, you were required to submit full recipe details. That is normally optional. With that in mind, I was interested to see one of the judges is in the process of opening up a nano and I wondered if that might be a conflict of interest? At least for something like Garrison, Big Spruce, Hammond River etc. they are up front and say they are looking for a beer to brew commercially.

Yah i found it odd that her creds say nano planned in TO.. clearly not indicative of any beer knowledge what so ever. The score sheet she did on my stout was so similar to this. At a competition level there needs to be someone that says it isn't right that we are going to send a score sheet like this back to someone who paid to put beer in this competition.

For aroma hops in this IPA it was Simcoe, Cascade, Citra and Perle (@15)

I do a hop whirlpool with the cascade and citra at flameout so most of what you get is the citrus flavour but the simcoe does add a bit of punch as well.

The 1/12 and the 1/20 are unreal... like Liam said unless it was water in the DIPA category i'm not sure why a judge would ever score a beer like that

[gm-]

The 1/12 and 1/20 is ridiculous, hope the organizers will learn to remove stuff like this from the judging in the future

[mcgster]

The 1/12 and 1/20 is ridiculous, hope the organizers will learn to remove stuff like this from the judging in the future

Yah that is my hope too. I mean the first year for a comp will always be rocky i'm sure. As long as they recognize the issues and dress them for next year i wouldn't have a problem entering again.