Rely command - assessing reliability for each code separately

Mon Aug 5 13:56:41 UTC 2024

Hi Leonid,

Thank you so much for this detailed response!

Yes, that code worked! This code (rely +c -t* +t%spa +t*MOT +d Marissa1.cha
Marissa2.cha +slowq-s) captures reliability for the lowq-s code, and I
added all lowq codes (rely +c -t* +t%spa +t*MOT +d Marissa1.cha
Marissa2.cha +slowq-s +lowq-m + slowq-e) to gather reliability for lowq
overall.

Thanks!
Best,
Marissa

On Fri, Aug 2, 2024 at 10:25 PM Leonid Spektor <spektor at andrew.cmu.edu>
wrote:

> Hi Marissa,
>
> I am not sure if I understand correctly what you want. I have made two
> sample files. Here is the first one:
>
> *MOT: child's word.
> %spa: lowq-s
> *MOT: something other .
> %spa: lowq-s
> *MOT: child's word.
> %spa: refq-s
> *MOT: something other .
> %spa: refq-e
>
> and here is the second one:
>
> *MOT: child's word.
> %spa: lowq-s
> *MOT: something other .
> %spa: refq-e
> *MOT: child's word.
> %spa: lowq-s
> *MOT: something other .
> %spa: refq-m
>
> -----------------------------------
> if I ran the following command, then I get result:
> > rely +c -t* +t%spa +t*MOT +d Marissa1.cha Marissa2.cha
>
> Marissa1.cha: # utterances examined: 4
> Marissa2.cha: # utterances examined: 4
> % of all utterances examined with matching codes %: 100.0000
>
> Marissa1.cha: # words examined: 4
> Marissa2.cha: # words examined: 4
> # words (tokens, codes) matched: 3
> % of all words examined with matching codes %: 75.0000
>
> I believe it is a correct result, because all codes except for "refq-s"
> and "refq-m" match. Please note that the match is for the whole file and
> not based on what speaker utterances those code are associated with.
>
> -----------------------------------
> if I ran the following command, then I get result:
> > rely +c -t* +t%spa +t*MOT +d Marissa1.cha Marissa2.cha +slowq-s
>
> Marissa1.cha: total lowq-s: 2
> Marissa2.cha: total lowq-s: 2
> total lowq-s matched: 2
> total lowq-s %: 100.0000
>
> In this case RELY only looks at "lowq-s" codes and those two codes per
> file match exactly between two files.
>
> -----------------------------------
> if I ran the following command, then I get result:
> > rely +c -t* +t%spa +t*MOT +d Marissa1.cha Marissa2.cha +s*-s
>
> Marissa1.cha: total *-s: 3
> Marissa2.cha: total *-s: 2
> total *-s matched: 2
> total *-s %: 66.6667
>
>  Again there are three codes with -s in the first file  and there are only
> two codes in the second file.
>
>
> Leonid.
> -----------------------------------
> I am attaching my two sample files here. If this is not what you wanted,
> then please explain in more details and maybe give me an example of what
> you are trying to achieve. Please email to me directly at
> spektor at andrew.cmu.edu.
>
>
> Leonid.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "chibolts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe
> <https://urldefense.com/v3/__https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe__;!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2yRSwkMzE$>
> .
> To unsubscribe from this group and all its topics, send an email to
> chibolts+unsubscribe at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511%40andrew.cmu.edu
> <https://urldefense.com/v3/__https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511*40andrew.cmu.edu?utm_medium=email&utm_source=footer__;JQ!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2ybxQcp3U$>
> .
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "chibolts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe
> <https://urldefense.com/v3/__https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe__;!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2yRSwkMzE$>
> .
> To unsubscribe from this group and all its topics, send an email to
> chibolts+unsubscribe at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511%40andrew.cmu.edu
> <https://urldefense.com/v3/__https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511*40andrew.cmu.edu?utm_medium=email&utm_source=footer__;JQ!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2ybxQcp3U$>
> .
>
> On Aug 2, 2024, at 18:34, Marissa Castellana <macaste8 at asu.edu> wrote:
>
> Hello,
>
> I am trying to assess inter-rater reliability using the RELY command for
> each of the four main codes (lowq, refq, recq, and infq) on the dependent
> tier (%spa) in our transcripts. Each code was also denoted with a "-s",
> "-m", or "-e" depending on the language used. For example, an utterance may
> be tagged as "lowq-s", "lowq-m", or "lowq-e".
>
> However, my code (*rely +c  -t* +t%spa +t*MOT +d*) seems to capture *overall
> *reliability instead of reliability *per code*. I have tried adding
> *+s"lowq"* and also attempted adding the more specific sub-code
> *+s"lowq-e"*, but these are not producing the intended results. Does
> anyone have any advice or potential solutions so that I can gather
> reliability per code rather than overall?
>
> Thank you!
> Best,
> Marissa
>
> --
> You received this message because you are subscribed to the Google Groups
> "chibolts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to chibolts+unsubscribe at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/chibolts/db4a7ef9-4541-4d8d-900a-d4c49d973ba7n%40googlegroups.com
> <https://urldefense.com/v3/__https://groups.google.com/d/msgid/chibolts/db4a7ef9-4541-4d8d-900a-d4c49d973ba7n*40googlegroups.com?utm_medium=email&utm_source=footer__;JQ!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2yrwKdRxg$>
> .
>
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "chibolts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe
> <https://urldefense.com/v3/__https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe__;!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2yRSwkMzE$>
> .
> To unsubscribe from this group and all its topics, send an email to
> chibolts+unsubscribe at googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511%40andrew.cmu.edu
> <https://urldefense.com/v3/__https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511*40andrew.cmu.edu?utm_medium=email&utm_source=footer__;JQ!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2ybxQcp3U$>
> .
>

-- 
You received this message because you are subscribed to the Google Groups "chibolts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chibolts+unsubscribe at googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/chibolts/CAF_YSzoSEJ0SCvHhn5kAp8j2Tmv9%3DYweXhWiE4k1FCF0WMuF1w%40mail.gmail.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/chibolts/attachments/20240805/e0ea77f3/attachment.htm>