I am aware of two methods (though there may be other methods) to calculate
the level of agreement between two persons who rated the same elements on
the same constructs. These methods which do not give the same results are:
(1) A difference grid (the absolute difference for each cell, summed by row
and column)
(2) Transforming ratings into a "disagreement score", e.g., a 0 if there was
agreement, 1 if person
used 4 and the other 1 or 2, and 2 if they were on opposite ends of the scale.
I also considered:
Bannister's consistency score
the possibility of joining the two grids and performing a cluster analysis etc
comparison on mean ratings for certain elements/constructs
Is there a methodologically preferable way of calculating difference (or
alternatively sameness).
Any advice is appreciated,
Bob Green
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%