<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Dear Karin,<div class=""> Looking now at that section of the CLAN manual, I see that it had not been properly updated to include both the +dm1 and +dN switches. Here is the revised section:</div><div class=""><br class=""></div><div class="">The second function of the RELY command is to evaluate the
correctness of a student transcript against a master transcript. The student is
given a version of the complete master transcript without the coding tier and
the goal is to add codes. In this case, it is assumed that the coding in the
master transcript is fully correct. The question is whether the student's codes
are correct. To specify that the first file given is the master, you use the
+dm1 switch. In addition, to compute
Cohen's kappa you need to add the +dN switch. In this example, we can say that
there are only two levels to the category, making the +dN switch to be
+d2. However, the number used would depend
on the actual levels of the coding system.</div><div class=""><p class="MsoNormal"><o:p class=""></o:p></p><p class="output"><span lang="ES-TRAD" class="">rely +d2 +dm1 sample.cha samplea.cha +t%spa<o:p class=""></o:p></span></p><p class="MsoNormal" style="text-indent:0in">The output from this command
provides two counts: precision and accuracy. These are the two dimensions used in fields
such as computational linguistics to evaluate correctness. Precision is the percentage of fields in the
master file that are matched by the student. Accuracy is the percentage of
fields in the student file that are matched to the master. A failure in
precision occurs when the student misses a code in the master file. A failure in accuracy occurs when the student
inserts a code that is not found in the master. If a researcher is interested
in going beyond these two scores, they can also compute the F score which is
the harmonic mean of precision and accuracy.
By adding the +d2 switch, the command also produces Cohen's kappa.<o:p class=""></o:p></p>
<!--EndFragment--><div><br class=""></div><div>--Brian MacWhinney</div><div><br class=""><blockquote type="cite" class=""><div class="">On Jun 11, 2020, at 5:17 AM, Karin Edlund <<a href="mailto:karin.eva.edlund@gmail.com" class="">karin.eva.edlund@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Hi,<div class="">What is the accurate command for RELY and computing percentage agreement on a dependant tier (i.e. %cod)?</div><div class="">I have tried </div><div class="">rely +d sample.cha samplea.cha +t%cod -t*</div><div class=""><br class=""></div><div class="">Also, what is the command for calculating Cohen's kappa on %cod tier?</div><div class="">I have tried with +dN, but I can't get it to work unfortunately.</div><div class=""><br class=""></div><div class="">Happy to receive advice on how to solve this.</div><div class=""><br class=""></div><div class="">Best Regards,</div><div class="">Karin Edlund</div></div><div class=""><br class="webkit-block-placeholder"></div>
-- <br class="">
You received this message because you are subscribed to the Google Groups "chibolts" group.<br class="">
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:chibolts+unsubscribe@googlegroups.com" class="">chibolts+unsubscribe@googlegroups.com</a>.<br class="">
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/chibolts/CANkUE2xW3-jbmfqNXyCVLAwZOh8dpCccdTeEWqujDsJQqTgH%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer" class="">https://groups.google.com/d/msgid/chibolts/CANkUE2xW3-jbmfqNXyCVLAwZOh8dpCccdTeEWqujDsJQqTgH%3DQ%40mail.gmail.com</a>.<br class="">
</div></blockquote></div><br class=""></div></body></html>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups "chibolts" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:chibolts+unsubscribe@googlegroups.com">chibolts+unsubscribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/chibolts/FF171C4D-A67B-4752-B054-98CD31394E58%40andrew.cmu.edu?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/chibolts/FF171C4D-A67B-4752-B054-98CD31394E58%40andrew.cmu.edu</a>.<br />