RELY

[Karin Edlund]

Brian,

Thank you so much again! I will try my best to get the commands to work.

Best Regards,
Karin Edlund

lör 13 juni 2020 kl. 23:32 skrev Brian MacWhinney <macw at andrew.cmu.edu>:

>
>
> On Jun 13, 2020, at 6:34 AM, Karin Edlund <karin.eva.edlund at gmail.com>
> wrote:
>
> Hi Brian,
> Thank you for your input on the RELY function.
> What I would like to do is the following:
>
> 1) Check reliability on percentage agreement for word and utterance level
> on each of 15 transcriptions comprising 5 min each.
> I was thinking of using the third function of RELY - the overall match on
> the main line between two transcripts - rely +d sample.cha samplea.cha
> This would be only to check the transcription, not the coding.
> I would say the two transcribers are on the same level regarding
> transcribing, thus perhaps it is not a case of checking correctness of a
> student against a master transcript.
> Or is the relation master-student more at matter of what data counts as
> the primary data?
>
>
> Yes, that will work for that comparison.
>
>
> 2) Compute Cohen's kappa for the coding. Two coders. I have five different
> categories for coding. I am not using %spa, but %cod.
> Would this be the right code for that:
> rely +d5 sample.cha samplea.cha +t%cod
>
> what does dm1 stand for in the RELY code you suggested?
>
>
> You still need +dm1 to tell CLAN that you consider the first file as the
> master.
>
> --Brian
>
>
> Perhaps I am mixing things up, but overall I am trying to check percentage
> agreement for word and utterance level in the transcriptions between two
> transcribers and secondly I am computing cohen's kappa for the coding.
> Seeking for the best RELY function and its respective code to do this.
>
> Best Regards,
> Karin Edlund
>
> Den tors 11 juni 2020 kl 18:41 skrev Brian MacWhinney <macw at andrew.cmu.edu
> >:
>
>> Dear Karin,
>>     You need to provide a number for N, based on the number of categories
>> in your coding system.  I am assuming that you are using RELY as described
>> in the second function in the section on RELY in the CLAN manual.  In the
>> manual, the suggested command uses +d1 which doesn't make sense.  If you
>> have two categories, you need +d2.  Then, for that example in the manual.
>> you get Cohen's kappa of 0.92.  Of course, all of this depends on the
>> actual shape of your coding system, and this little example can't
>> illustrate that.  However, I will at least fix that error in the manual.
>>
>> --Brian MacWhinney
>>
>> On Jun 11, 2020, at 5:17 AM, Karin Edlund <karin.eva.edlund at gmail.com>
>> wrote:
>>
>> Hi,
>> What is the accurate command for RELY and computing
>> percentage agreement on a dependant tier (i.e. %cod)?
>> I have tried
>> rely +d sample.cha samplea.cha +t%cod -t*
>>
>> Also, what is the command for calculating Cohen's kappa on %cod tier?
>> I have tried with +dN, but I can't get it to work unfortunately.
>>
>> Happy to receive advice on how to solve this.
>>
>> Best Regards,
>> Karin Edlund
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "chibolts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to chibolts+unsubscribe at googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/chibolts/CANkUE2xW3-jbmfqNXyCVLAwZOh8dpCccdTeEWqujDsJQqTgH%3DQ%40mail.gmail.com
>> <https://groups.google.com/d/msgid/chibolts/CANkUE2xW3-jbmfqNXyCVLAwZOh8dpCccdTeEWqujDsJQqTgH%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "chibolts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to chibolts+unsubscribe at googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/chibolts/C233654B-250D-4B81-B2DC-F2005E936177%40andrew.cmu.edu
>> <https://groups.google.com/d/msgid/chibolts/C233654B-250D-4B81-B2DC-F2005E936177%40andrew.cmu.edu?utm_medium=email&utm_source=footer>
>> .
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "chibolts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chibolts+unsubscribe at googlegroups.com.
>
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/chibolts/CANkUE2y-xD6aGFLzh26e%2Bvx185OJwvSKdAnB4j_%2BTGfmTVSJ_w%40mail.gmail.com
> <https://groups.google.com/d/msgid/chibolts/CANkUE2y-xD6aGFLzh26e%2Bvx185OJwvSKdAnB4j_%2BTGfmTVSJ_w%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "chibolts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chibolts+unsubscribe at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/chibolts/116DA890-299A-4EBA-B7C1-DFFB5C22AD1D%40andrew.cmu.edu
> <https://groups.google.com/d/msgid/chibolts/116DA890-299A-4EBA-B7C1-DFFB5C22AD1D%40andrew.cmu.edu?utm_medium=email&utm_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups "chibolts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/CANkUE2zrq%3Dukcn453Pyc47_u1_tEAjBiWeZ2jSw12BsEaLch%2Bw%40mail.gmail.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/chibolts/attachments/20200615/8c8d2568/attachment.htm>