measuring change

bob green (bgreen@dyson.brisnet.org.au)
Wed, 26 May 1999 22:15:04 +1000

Further to the recent posts about assessing change in parent perceptions and
also being asked at work about what analysis could be used to evaluate
whether a small group program resulted in change (essentially all the advice
I gathered was that little statistical analysis could be performed other
than examining mean ratings), I considered the problem of examining change
via a repertory grid would raise similar issues.

Recently, I asked Chris Evans about a 1991 paper he wrote which compared the
construing of a group of women referred to counselling for an eating
disorder with a group of women, without such problems. My questions related
his use of inter-element distances and construct-element angular distances
rather than the use of mean ratings. Chris suggested mean ratings can be
problematic when positive poles in grids are not aligned, and when there are
extreme ratings.

My first question concerns whether anyone knows of literature to support the
use of distance measures vs mean ratings. Additionally, distance measures
can be abstract to explain. A MDS analysis which uses as data distances
between cities and then compares the spatial plot vs a map, is one thing.
Explaining how ratings are transformed into distances and the assumption of
similarity based on proximity, is another.

Winter in his Personal Construct Psychology in Clinical Practice, describes
several methods to examine change:

distance between elements
cosine or correlation between element or construct
size of the sum of squares of an element compared to the average sum of squares

Question 2: I am aware of only a few papers which discuss the relative
merits of some of these approaches, e.g.,

Mackay N (1992) Identification, Reflcetion and Correlation. IJPCP, 5, 53-77.
Bell R (1988) Theory-Apprporiate analysis of repertory Grid data. IJPCP, 1,
101-118.

Both these papers discuss limitations of correlational approaches, and
Mackay recommended an approach based on sum of squares.

any further suggestions or comments?

In addition to rearranging data (e.g joining two grids which shared the same
elements), Patrick Slater raised the issue of consistency, i.e., when is
difference the result of inconsistency and when it is a result of change.
One of his suggestions was to examine whether the pattern of correlations
remained the same, even though the evluation of two elements may have
changed. Also, in a worked example, Slater compares two grids in terms of:
correlation between elements, mean ratings and sum of squares.

A variation of Slater's INGRID program was used by Bailey and Sims, which
allowed the examination of distance between elements in terms, "of the
proportion of elexpected distance, which is a function of the total
variation of the grid and the number of elements".

Any comments or alternative thoughts/options are welcomed,

regards,

Bob Green

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%