RE: Hello, and questions (long!)

Wed, 23 Feb 1994 10:44:03 EST

Second problem: I have these interesting-looking grids, the assessors have
gone over them again and verified that this is indeed what they feel, and
found explanations in them to the qualms they had regarding the
(previously made) holistic assessment of some of the performances, but: I
don't know how to analyse them. I have run correlations, they are
mostly very high, .7 to .9, some individual ones at .5-range. Naturally,
factor analysis produces one factor. This should mean there's only one
construct operating. In that case, I would call this construct language
ability, rather than any of the individual labels elicited. But what of
the labels, should I say they don't mean anything? But they do! They
are very real to the assessors, and clearly separable, rate of speaking
for instance, from extent of vocabulary, which is again different from
structural knowledge. It just so happens that language is an integrated
skill, as ability grows, all of these features improve, some more
linearlythan others. But this doesn't mean the features don't
exist, not to me at least. Will I ever be able to do more than label

All it seems to imply is that the assessor's grids are defining a high
dimensional world than the candidates are presenting. In other words assessors
can conceive of a candidate being good on evaluative construct 1 but not good
on evaulative construct 2. However there are none or very few candidates who
actually present with that combination. You could contrast the politically
correct multidimensionality of the assessors with the one-dimensional reality
of the candidates. I think it is pointless the go the full empirical mile
and start to argue that the assessor's view of the world does not exist
because candidates do not fill all its parts.
The only way to make assessor's multi-dimensionallity to work is to have them
think in (dare I say) normative terms. Compared with someone who has achieved
general ability X, this person is above or below on a series of constructs.
Then perhaps you would gain something multi-dimensional. In other words you
are needing partial correlations corrected for candidate ability. However
rather than trying a technical statistical fix, try changing the judgements.
Dr Stephen K Tagg
Department of Marketing
(been analyzing grids for 20 years - mainly with MDS procedures...)