[Corpora-List] Kappa coefficient for more than two annotators
Adam Kilgarriff
adam at lexmasterclass.com
Tue Jul 18 17:29:47 UTC 2006
Narjes,
Don't assume it will get you an informative result. Kappa depends on what a
random guesser does, but the random guesser's results depend on the
probability function you assume. It is easy to use kappa to spread more
confusion that light. See section 5 of
http://www.kilgarriff.co.uk/Publications/2003-K-EACLBuda-noburo.pdf
Adam
-----Original Message-----
From: owner-corpora at lists.uib.no [mailto:owner-corpora at lists.uib.no] On
Behalf Of Narjes Boufaden
Sent: 18 July 2006 17:35
To: corpora
Subject: [Corpora-List] Kappa coefficient for more than two annotators
Hi everybody,
I'm looking for a perl script that estimates the Kappa coefficient for
data coming from more than two annotators and with multiple classes.
Thanks a lot for any help,
Narjès.
More information about the Corpora
mailing list