<div dir="ltr">Hi Leonid,<div><br></div><div>Thank you so much for this detailed response! </div><div><br></div><div>Yes, that code worked! This code (rely +c -t* +t%spa +t*MOT +d Marissa1.cha Marissa2.cha +slowq-s) captures reliability for the lowq-s code, and I added all lowq codes (rely +c -t* +t%spa +t*MOT +d Marissa1.cha Marissa2.cha +slowq-s +lowq-m + slowq-e) to gather reliability for lowq overall.</div><div><br></div><div>Thanks!</div><div>Best,</div><div>Marissa</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 2, 2024 at 10:25 PM Leonid Spektor <<a href="mailto:spektor@andrew.cmu.edu">spektor@andrew.cmu.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>Hi Marissa,<div><br></div><div>I am not sure if I understand correctly what you want. I have made two sample files. Here is the first one:</div><div><br></div><div>*MOT:<span style="white-space:pre-wrap"> </span>child's word.</div><div>%spa:<span style="white-space:pre-wrap"> </span>lowq-s</div><div>*MOT:<span style="white-space:pre-wrap"> </span>something other .</div><div>%spa:<span style="white-space:pre-wrap"> </span>lowq-s</div><div>*MOT:<span style="white-space:pre-wrap"> </span>child's word.</div><div>%spa:<span style="white-space:pre-wrap"> </span>refq-s</div><div>*MOT:<span style="white-space:pre-wrap"> </span>something other .</div><div>%spa:<span style="white-space:pre-wrap"> </span>refq-e</div><div><br></div><div>and here is the second one:</div><div><br></div><div>*MOT:<span style="white-space:pre-wrap"> </span>child's word.</div><div>%spa:<span style="white-space:pre-wrap"> </span>lowq-s</div><div>*MOT:<span style="white-space:pre-wrap"> </span>something other .</div><div>%spa:<span style="white-space:pre-wrap"> </span>refq-e</div><div>*MOT:<span style="white-space:pre-wrap"> </span>child's word.</div><div>%spa:<span style="white-space:pre-wrap"> </span>lowq-s</div><div>*MOT:<span style="white-space:pre-wrap"> </span>something other .</div><div>%spa:<span style="white-space:pre-wrap"> </span>refq-m</div><div><br></div><div><div>-----------------------------------</div><div><div></div></div></div><div>if I ran the following command, then I get result:</div><div><div>> rely +c -t* +t%spa +t*MOT +d Marissa1.cha Marissa2.cha</div><div><br></div><div>Marissa1.cha: # utterances examined: 4</div><div>Marissa2.cha: # utterances examined: 4</div><div>% of all utterances examined with matching codes %: 100.0000</div><div><br></div><div>Marissa1.cha: # words examined: 4</div><div>Marissa2.cha: # words examined: 4</div><div># words (tokens, codes) matched: 3</div><div>% of all words examined with matching codes %: 75.0000</div></div><div><br></div><div>I believe it is a correct result, because all codes except for "refq-s" and "refq-m" match. Please note that the match is for the whole file and not based on what speaker utterances those code are associated with.</div><div><br></div><div>-----------------------------------</div><div><div></div></div><div>if I ran the following command, then I get result:</div><div><div></div><div>> rely +c -t* +t%spa +t*MOT +d Marissa1.cha Marissa2.cha +slowq-s</div></div><div><div><br></div><div>Marissa1.cha: total lowq-s: 2</div><div>Marissa2.cha: total lowq-s: 2</div><div>total lowq-s matched: 2</div><div>total lowq-s %: 100.0000</div></div><div><br></div><div>In this case RELY only looks at "lowq-s" codes and those two codes per file match exactly between two files.</div><div><br></div><div>-----------------------------------</div><div><div><div>if I ran the following command, then I get result:</div><div></div><div>> rely +c -t* +t%spa +t*MOT +d Marissa1.cha Marissa2.cha +s*-s</div></div></div><div><div><br></div><div>Marissa1.cha: total *-s: 3</div><div>Marissa2.cha: total *-s: 2</div><div>total *-s matched: 2</div><div>total *-s %: 66.6667</div></div><div><br></div><div> Again there are three codes with -s in the first file and there are only two codes in the second file.<br id="m_-5398541704185534054lineBreakAtBeginningOfMessage"><div>
<br><br style="color:rgb(0,0,0);font-family:Arial;font-size:16px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><span style="color:rgb(0,0,0);font-family:Arial;font-size:16px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">Leonid.</span>
</div>
<div>
<div>-----------------------------------</div><div><div></div></div>I am attaching my two sample files here. If this is not what you wanted, then please explain in more details and maybe give me an example of what you are trying to achieve. Please email to me directly at <a href="mailto:spektor@andrew.cmu.edu" target="_blank">spektor@andrew.cmu.edu</a>.</div><div><br></div><div><br style="color:rgb(0,0,0);font-family:Arial;font-size:16px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><span style="color:rgb(0,0,0);font-family:Arial;font-size:16px;font-style:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline">Leonid.</span></div><div></div></div></div>
<p></p>
-- <br>
You received this message because you are subscribed to a topic in the Google Groups "chibolts" group.<br>
To unsubscribe from this topic, visit <a href="https://urldefense.com/v3/__https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe__;!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2yRSwkMzE$" target="_blank">https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe</a>.<br>
To unsubscribe from this group and all its topics, send an email to <a href="mailto:chibolts+unsubscribe@googlegroups.com" target="_blank">chibolts+unsubscribe@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://urldefense.com/v3/__https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511*40andrew.cmu.edu?utm_medium=email&utm_source=footer__;JQ!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2ybxQcp3U$" target="_blank">https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511%40andrew.cmu.edu</a>.<br>
<div><div><div></div></div></div>
<p></p>
-- <br>
You received this message because you are subscribed to a topic in the Google Groups "chibolts" group.<br>
To unsubscribe from this topic, visit <a href="https://urldefense.com/v3/__https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe__;!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2yRSwkMzE$" target="_blank">https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe</a>.<br>
To unsubscribe from this group and all its topics, send an email to <a href="mailto:chibolts+unsubscribe@googlegroups.com" target="_blank">chibolts+unsubscribe@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://urldefense.com/v3/__https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511*40andrew.cmu.edu?utm_medium=email&utm_source=footer__;JQ!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2ybxQcp3U$" target="_blank">https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511%40andrew.cmu.edu</a>.<br>
<div><div><div></div>
<div><br><blockquote type="cite"><div>On Aug 2, 2024, at 18:34, Marissa Castellana <<a href="mailto:macaste8@asu.edu" target="_blank">macaste8@asu.edu</a>> wrote:</div><br><div>Hello,<div><br></div><div>I am trying to assess inter-rater reliability using the RELY command for each of the four main codes (lowq, refq, recq, and infq) on the dependent tier (%spa) in our transcripts. Each code was also denoted with a "-s", "-m", or "-e" depending on the language used. For example, an utterance may be tagged as "lowq-s", "lowq-m", or "lowq-e".<br></div><div><br></div><div>However, my code (<b>rely +c -t* +t%spa +t*MOT +d</b>) seems to capture <i>overall </i>reliability instead of reliability <i>per code</i>. I have tried adding <b>+s"lowq"</b> and also attempted adding the more specific sub-code <b>+s"lowq-e"</b>, but these are not producing the intended results. Does anyone have any advice or potential solutions so that I can gather reliability per code rather than overall?</div><div><br></div><div>Thank you!</div><div>Best,</div><div>Marissa</div><div><br></div>
-- <br>
You received this message because you are subscribed to the Google Groups "chibolts" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:chibolts+unsubscribe@googlegroups.com" target="_blank">chibolts+unsubscribe@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://urldefense.com/v3/__https://groups.google.com/d/msgid/chibolts/db4a7ef9-4541-4d8d-900a-d4c49d973ba7n*40googlegroups.com?utm_medium=email&utm_source=footer__;JQ!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2yrwKdRxg$" target="_blank">https://groups.google.com/d/msgid/chibolts/db4a7ef9-4541-4d8d-900a-d4c49d973ba7n%40googlegroups.com</a>.<br>
</div></blockquote></div><br></div></div>
<p></p>
-- <br>
You received this message because you are subscribed to a topic in the Google Groups "chibolts" group.<br>
To unsubscribe from this topic, visit <a href="https://urldefense.com/v3/__https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe__;!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2yRSwkMzE$" target="_blank">https://groups.google.com/d/topic/chibolts/NTR16qI-RIc/unsubscribe</a>.<br>
To unsubscribe from this group and all its topics, send an email to <a href="mailto:chibolts+unsubscribe@googlegroups.com" target="_blank">chibolts+unsubscribe@googlegroups.com</a>.<br>
To view this discussion on the web visit <a href="https://urldefense.com/v3/__https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511*40andrew.cmu.edu?utm_medium=email&utm_source=footer__;JQ!!IKRxdwAv5BmarQ!YvfvoP7GneYaGzB-V4h2_V_cETmAMerJZqDfxa7i-gQxPNmsyWrIcrzQ3ACRyzzUyEOYPvoYbSZLS-xvzE2ybxQcp3U$" target="_blank">https://groups.google.com/d/msgid/chibolts/EACA6EF9-AE62-48CE-9FB4-7CB28F239511%40andrew.cmu.edu</a>.<br>
</blockquote></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups "chibolts" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:chibolts+unsubscribe@googlegroups.com">chibolts+unsubscribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/chibolts/CAF_YSzoSEJ0SCvHhn5kAp8j2Tmv9%3DYweXhWiE4k1FCF0WMuF1w%40mail.gmail.com?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/chibolts/CAF_YSzoSEJ0SCvHhn5kAp8j2Tmv9%3DYweXhWiE4k1FCF0WMuF1w%40mail.gmail.com</a>.<br />