RELY

[Brian MacWhinney]

> On Jun 13, 2020, at 6:34 AM, Karin Edlund <karin.eva.edlund at gmail.com> wrote:
> 
> Hi Brian,
> Thank you for your input on the RELY function.
> What I would like to do is the following:
> 
> 1) Check reliability on percentage agreement for word and utterance level on each of 15 transcriptions comprising 5 min each.
> I was thinking of using the third function of RELY - the overall match on the main line between two transcripts - rely +d sample.cha samplea.cha
> This would be only to check the transcription, not the coding. 
> I would say the two transcribers are on the same level regarding transcribing, thus perhaps it is not a case of checking correctness of a student against a master transcript. 
> Or is the relation master-student more at matter of what data counts as the primary data?

Yes, that will work for that comparison.
> 
> 2) Compute Cohen's kappa for the coding. Two coders. I have five different categories for coding. I am not using %spa, but %cod.
> Would this be the right code for that:
> rely +d5 sample.cha samplea.cha +t%cod 
> 
> what does dm1 stand for in the RELY code you suggested?

You still need +dm1 to tell CLAN that you consider the first file as the master.  

--Brian
> 
> Perhaps I am mixing things up, but overall I am trying to check percentage agreement for word and utterance level in the transcriptions between two transcribers and secondly I am computing cohen's kappa for the coding. 
> Seeking for the best RELY function and its respective code to do this. 
> 
> Best Regards,
> Karin Edlund
> 
> Den tors 11 juni 2020 kl 18:41 skrev Brian MacWhinney <macw at andrew.cmu.edu <mailto:macw at andrew.cmu.edu>>:
> Dear Karin,
>     You need to provide a number for N, based on the number of categories in your coding system.  I am assuming that you are using RELY as described in the second function in the section on RELY in the CLAN manual.  In the manual, the suggested command uses +d1 which doesn't make sense.  If you have two categories, you need +d2.  Then, for that example in the manual. you get Cohen's kappa of 0.92.  Of course, all of this depends on the actual shape of your coding system, and this little example can't illustrate that.  However, I will at least fix that error in the manual.
> 
> --Brian MacWhinney
> 
>> On Jun 11, 2020, at 5:17 AM, Karin Edlund <karin.eva.edlund at gmail.com <mailto:karin.eva.edlund at gmail.com>> wrote:
>> 
>> Hi,
>> What is the accurate command for RELY and computing percentage agreement on a dependant tier (i.e. %cod)?
>> I have tried 
>> rely +d sample.cha samplea.cha +t%cod -t*
>> 
>> Also, what is the command for calculating Cohen's kappa on %cod tier?
>> I have tried with +dN, but I can't get it to work unfortunately.
>> 
>> Happy to receive advice on how to solve this.
>> 
>> Best Regards,
>> Karin Edlund
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups "chibolts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+unsubscribe at googlegroups.com <mailto:chibolts+unsubscribe at googlegroups.com>.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/CANkUE2xW3-jbmfqNXyCVLAwZOh8dpCccdTeEWqujDsJQqTgH%3DQ%40mail.gmail.com <https://groups.google.com/d/msgid/chibolts/CANkUE2xW3-jbmfqNXyCVLAwZOh8dpCccdTeEWqujDsJQqTgH%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups "chibolts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+unsubscribe at googlegroups.com <mailto:chibolts+unsubscribe at googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/C233654B-250D-4B81-B2DC-F2005E936177%40andrew.cmu.edu <https://groups.google.com/d/msgid/chibolts/C233654B-250D-4B81-B2DC-F2005E936177%40andrew.cmu.edu?utm_medium=email&utm_source=footer>.
> 
> -- 
> You received this message because you are subscribed to the Google Groups "chibolts" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+unsubscribe at googlegroups.com <mailto:chibolts+unsubscribe at googlegroups.com>.
> To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/CANkUE2y-xD6aGFLzh26e%2Bvx185OJwvSKdAnB4j_%2BTGfmTVSJ_w%40mail.gmail.com <https://groups.google.com/d/msgid/chibolts/CANkUE2y-xD6aGFLzh26e%2Bvx185OJwvSKdAnB4j_%2BTGfmTVSJ_w%40mail.gmail.com?utm_medium=email&utm_source=footer>.

-- 
You received this message because you are subscribed to the Google Groups "chibolts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/116DA890-299A-4EBA-B7C1-DFFB5C22AD1D%40andrew.cmu.edu.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/chibolts/attachments/20200613/e3eb87a1/attachment.htm>