[Corpora-List] How to calculate kappa statistic!

[Deepa Gupta]

Hello everyone, 
 
I am not clear how to calculate kappa statistic for word alignment task when two annotators(A1 and A2) output are given.  
let's take 
A1 -- (1 1) (2 5) (3 3) (3 4 p) (4 2) (5 6)
A2 - (1 1) (2 5 p) (3 3) (4 2) (5 6) (0 4 p).
 
here (1 1 s/p) --> (source word, target word, sure-link/probable link)
only probable links are mapped in both, otherwise all are sure links. 
 
please explain the kappa coff. using above example.
 
I have checked all relevant papers, but still i  am not clear.
 
Thanks
deepa
 


      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20100804/94feee4b/attachment.htm>
-------------- next part --------------
_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora