Corpora: kappa statistics

[Constantin Orasan]

Dear readers,

I was wondering if it is possible to compute kappa statistics for the
interannotator agreement in cases when the number of classes is not
known apriori (e.g. for coreference annotation the number of chains is
not known before the annotation is finished. Does anyone know how the
agreement can be computed in such a case?)

Thank you,
Constantin

Computational Linguistic Group
University of Wolverhampton
http://www.wlv.ac.uk/~in6093