Quadratic Cohen Kappa

Quadratic Cohen's Kappa is a variant of Cohen's Kappa that is often used in cases where the variable of interest is ordinal (i.e., the potential categories have a natural order, such as 'low', 'medium', and 'high'). Like Linear Cohen's Kappa, Quadratic Cohen's Kappa incorporates a weight matrix to account for the degree of disagreement between raters.

The distinction between Linear and Quadratic Cohen's Kappa is in how they calculate this degree of disagreement. Linear Cohen's Kappa weights disagreements in a linear fashion, meaning that each step away from the correct category is weighted equally. In contrast, Quadratic Cohen's Kappa weights disagreements quadratically, which places greater emphasis on categories that are further apart.

For example, with a five-point scale, a disagreement between a rating of 5 and 4 would be considered a smaller error than a disagreement between a rating of 5 and 3. With Quadratic Cohen's Kappa, the error for the disagreement between 5 and 3 would be four times larger than the error for the disagreement between 5 and 4, because it is twice as far away.

The choice between Linear and Quadratic Cohen's Kappa depends on the specifics of the problem at hand. Quadratic Cohen's Kappa is a stronger penalizer of larger disagreements, making it a useful choice when these larger disagreements are particularly undesirable.

Here's the formula for calculating the Weighted Kappa:

κ = (Po - Pe) / (1 - Pe)

But, unlike in regular Cohen's Kappa, in this case Po and Pe are calculated by summing the weighted proportions of agreements and expected agreements, respectively, across all categories.