RELY

[Karin Edlund]

Hi Brian,
Thank you for your input on the RELY function.
What I would like to do is the following:

1) Check reliability on percentage agreement for word and utterance level
on each of 15 transcriptions comprising 5 min each.
I was thinking of using the third function of RELY - the overall match on
the main line between two transcripts - rely +d sample.cha samplea.cha
This would be only to check the transcription, not the coding.
I would say the two transcribers are on the same level regarding
transcribing, thus perhaps it is not a case of checking correctness of a
student against a master transcript.
Or is the relation master-student more at matter of what data counts as the
primary data?

2) Compute Cohen's kappa for the coding. Two coders. I have five different
categories for coding. I am not using %spa, but %cod.
Would this be the right code for that:
rely +d5 sample.cha samplea.cha +t%cod

what does dm1 stand for in the RELY code you suggested?

Perhaps I am mixing things up, but overall I am trying to check percentage
agreement for word and utterance level in the transcriptions between two
transcribers and secondly I am computing cohen's kappa for the coding.
Seeking for the best RELY function and its respective code to do this.

Best Regards,
Karin Edlund

Den tors 11 juni 2020 kl 18:41 skrev Brian MacWhinney <macw at andrew.cmu.edu>:

> Dear Karin,
>     You need to provide a number for N, based on the number of categories
> in your coding system.  I am assuming that you are using RELY as described
> in the second function in the section on RELY in the CLAN manual.  In the
> manual, the suggested command uses +d1 which doesn't make sense.  If you
> have two categories, you need +d2.  Then, for that example in the manual.
> you get Cohen's kappa of 0.92.  Of course, all of this depends on the
> actual shape of your coding system, and this little example can't
> illustrate that.  However, I will at least fix that error in the manual.
>
> --Brian MacWhinney
>
> On Jun 11, 2020, at 5:17 AM, Karin Edlund <karin.eva.edlund at gmail.com>
> wrote:
>
> Hi,
> What is the accurate command for RELY and computing
> percentage agreement on a dependant tier (i.e. %cod)?
> I have tried
> rely +d sample.cha samplea.cha +t%cod -t*
>
> Also, what is the command for calculating Cohen's kappa on %cod tier?
> I have tried with +dN, but I can't get it to work unfortunately.
>
> Happy to receive advice on how to solve this.
>
> Best Regards,
> Karin Edlund
>
> --
> You received this message because you are subscribed to the Google Groups
> "chibolts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chibolts+unsubscribe at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/chibolts/CANkUE2xW3-jbmfqNXyCVLAwZOh8dpCccdTeEWqujDsJQqTgH%3DQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/chibolts/CANkUE2xW3-jbmfqNXyCVLAwZOh8dpCccdTeEWqujDsJQqTgH%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "chibolts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chibolts+unsubscribe at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/chibolts/C233654B-250D-4B81-B2DC-F2005E936177%40andrew.cmu.edu
> <https://groups.google.com/d/msgid/chibolts/C233654B-250D-4B81-B2DC-F2005E936177%40andrew.cmu.edu?utm_medium=email&utm_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups "chibolts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/CANkUE2y-xD6aGFLzh26e%2Bvx185OJwvSKdAnB4j_%2BTGfmTVSJ_w%40mail.gmail.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/chibolts/attachments/20200613/7e433a3d/attachment.htm>