In defense of Gearscore

Gevlon just posted an "I hate gearscore" rant, and while I agree with him on some levels, I disagree on the overall conclusion. Rather than just post a long comment on his blog, I'm posting my response here.

Let's ask the question sideways: who is hurt by  GearScore? In order to approach that question, I'm going to be making a couple of assumptions.

One: we are talking about a scenario wherein a raid leader needs to pug someone and puts a reasonable minimum GS on their pug requirements. By reasonable, I mean 5000 GS for ICC, which is easily accomplished by farming heroics for t9.232, or lower GS for lesser raids. I don't mean the 6.5k GS requirement for ANYTHING, or the 5k GS naxx run.

Two: while Gevlon has demonstrated that it's possible to kill a LOT of raid bosses in iLevel 200 blues, the reports also indicate that there's a lot of perseverance involved in those boss kills, on the order of ten to fifteen wipes per boss kill. Clearly a higher GS will assist in surviving and killing bosses. Skill is important, but skill coupled with GS is even better than skill alone.

Three: you're on a large enough server that there are high-GS raiders ready to PuG and fill in your slots. More on this later.

Okay, so we can look at two variables: GearScore and Skill (henceforth GS and S). Rather than make any fine distinctions, we'll just create two qualitative values: high and low (do a median split if you like, but really there's probably clusters in skill at least, and most likely in GS as well). Therefore we have four categories a Potential could fall into, and I've listed them in order of preference.

High-GS, High-S: we want these people in the raid. Gearscore does not prevent them from raiding. Once they pug with you once, if you're a good raid leader, you will note their performance and add them to the list of preferred puggers.

Low-GS, High-S: we'd like to have these people in the raid, because they probably can do a respectable job (at DPS at least) and probably know all the fights from first-hand alt experience or through thorough preparation at TankSpot and elsewhere. It kind of sucks that GS will serve as a barrier to their entry to the raid, but there is a bright side which is two-fold. Given the choice between High-GS, High-S and Low-GS, High-S, we'd rather have the former, and it only takes a week or so of heroic grinding for the highly skilled to go from Low-GS to High-GS. Less so if they're also good at the AH and can afford some choice BoE items (264 crafted boots/pants, battered hilt, etc). So while it sucks for both the raid and the raider that we exclude this category, such an exclusion is temporary and will be overcome later.

High-GS, Low-S: these people suck. They're the ones Gevlon is talking about. They grinded their way into t9 without learning their class. For example, just the other day I had a paladin healer put that stupid immunity bubble on me while I was tanking. The mobs I was tanking promptly aggro'd other people and there was a near wipe. How do you get to 80 without learning about that? Having these people in your raid will suck, and if you're a RL you need to boot them as soon as their incompetence becomes clear. On the other hand, if you're just trying to fill a DPS slot, having this dummy do 3k DPS is better than running with 24 people on some fights.

Low-GS, Low-S: the raid is better off without these people.

So, as you can see, using the GS metric to discriminate between Good and Bad raiders is subject to false positives (High-GS, Low-S) and false negatives (Low-GS, High-S). The false positive is in no way unique to the GearScore discrimination test. The only real way to determine skill is by extended interaction with the player. They could have been carried by their prior raids into their achievements, they could have paid someone else to farm their badges, they could have just taken over their older brother's WoW account when he went off to college. One job of the competent RL is to review pugger DPS on trash and their reactions to various fight mechanics (standing in the fire, etc) and boot people if they're not performing. The false negative, on the other hand, does kind of suck, but is easily remedied by the player in question: go get some tier gear and come back next week.

The next question is: given that GS is at worst, no worse than any other system that can discriminate potential raiders, what are its strengths? Strengths really come down to one thing: simplicity. While it's possible for a raid leader to check out a character's achievements using the armory (while maybe taking the pugger's word that their alt MrAwesomePants really is their alt and see is a Kingslayer), that's a lot of work and is no more likely to discriminate skill than a GS number.

Clearly, one can abuse the GS system, asking only for very high GS requirements to enter the raid. These raid leaders either have a very small number of slots to fill or have no real faith in their raid or their own ability to lead the raid and need the extra cushion provided by the 6k GS.

In an ideal world, we'd have a better discrimination test to easily detect skill without needing to watch someone raid. Even in that world, we'd still preferentially choose high-GS, high-S people over the low-GS, low-S people. GearScore may be a lousy way to pick puggers, but it seems like it's better than all the other lousy ways to pick puggers.