Cohen’s kappa can always be increased and decreased by combining categories

On Agreement Tables with Constant Kappa Values

Kappa coefficients are standard tools for summarizing the information in cross-classifications of two categorical variables with identical categories, here called agreement tables. When two categories are combined the kappa value usually either increases or decreases. There is a class of agreement tables for which the value of Cohen’s kappa remains constant when two categories are combined. It is shown that for this class of tables all special cases of symmetric kappa coincide and that the value of symmetric kappa is not affected by any partitioning of the categories.

Weighted Kappas for3×3Tables

Weighted kappa is a widely used statistic for summarizing inter-rater agreement on a categorical scale. For rating scales with three categories, there are seven versions of weighted kappa. It is shown analytically how these weighted kappas are related. Several conditional equalities and inequalities between the weighted kappas are derived. The analytical analysis indicates that the weighted kappas are measuring the same thing but to a different extent. One cannot, therefore, use the same magnitude guidelines for all weighted kappas.

A Preliminary Classification of Usage Measures in Information System Acceptance

In the light of a diverse body of disorganized usage measures available and the difficulty of building a cumulative research tradition, a literature review is conducted on system use in Information Systems (IS) Acceptance through the two main theories of Technology Adoption, the Technology Acceptance Model (TAM), and The Unified Theory of Use and Acceptance of Technology (UTAUT). The authors seek to understand how usage measures are being operationalised and proposed a preliminary classification of those measures that covers system and task aspects of use. A Q-Sort approach was taken to validate the authors’ classification scheme and the result indicates high inter-rater agreement. The ensuing classification is meant to help researchers in their choice of system use measures. This review also summarises the arguments for a multi-dimensional measure of use and establishes that omnibus measure such as frequency, volume and use/non-use hold prevalence. Finally, the authors provide recommendations for further research in the area of system use.