<div dir="ltr">Good evening,<br><br>I have a question about calculating the inter-rater reliability (Cohen's kappa coefficient) of codes in CLAN using the RELY command. <br><br>I have been using the command: rely +dN +c @ +t%num -t*, with N equaling the number of categories. Does this look correct?<br><br>I am unsure how to calculate the number of categories for my coding scheme. This table breaks down my coding scheme. There are three main codes ($OT, $RE, $NT). If $NT is selected, there are four subcodes, with differing quantities of options to choose from. A full $NT might look like: $NT:PAR:ALL:EME:P.<table cellpadding="0" class="gmail-ajC" style="border-spacing:0px;line-height:20px;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:14px"><tbody><tr class="gmail-ajv"><td colspan="2" tabindex="0" class="gmail-gL" style="vertical-align:top;width:auto;padding:2px 0px"><span class="gmail-gI" style="vertical-align:top"><br><table class="gmail-MsoTableGrid" border="1" cellspacing="0" cellpadding="0" style="border-collapse:collapse;border:none">
<tbody><tr>
<td width="120" style="width:89.75pt;border:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" align="center" style="margin:0in 0in 0.0001pt;text-align:center;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"><b>Main
code</b></p>
</td>
<td width="130" style="width:97.25pt;border-top:1pt solid windowtext;border-right:1pt solid windowtext;border-bottom:1pt solid windowtext;border-left:none;padding:0in 5.4pt">
<p class="MsoNormal" align="center" style="margin:0in 0in 0.0001pt;text-align:center;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"><b>Sub-Code
#1</b></p>
</td>
<td width="125" style="width:93.5pt;border-top:1pt solid windowtext;border-right:1pt solid windowtext;border-bottom:1pt solid windowtext;border-left:none;padding:0in 5.4pt">
<p class="MsoNormal" align="center" style="margin:0in 0in 0.0001pt;text-align:center;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"><b>Sub-Code
#2</b></p>
</td>
<td width="125" style="width:93.5pt;border-top:1pt solid windowtext;border-right:1pt solid windowtext;border-bottom:1pt solid windowtext;border-left:none;padding:0in 5.4pt">
<p class="MsoNormal" align="center" style="margin:0in 0in 0.0001pt;text-align:center;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"><b>Sub-Code
#3</b></p>
</td>
<td width="125" style="width:93.5pt;border-top:1pt solid windowtext;border-right:1pt solid windowtext;border-bottom:1pt solid windowtext;border-left:none;padding:0in 5.4pt">
<p class="MsoNormal" align="center" style="margin:0in 0in 0.0001pt;text-align:center;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"><b>Sub-Code
#4</b></p>
</td>
</tr>
<tr>
<td width="120" valign="top" style="width:89.75pt;border-right:1pt solid windowtext;border-bottom:1pt solid windowtext;border-left:1pt solid windowtext;border-top:none;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif">$OT</p>
</td>
<td width="130" valign="top" style="width:97.25pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"> </p>
</td>
<td width="125" valign="top" style="width:93.5pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"> </p>
</td>
<td width="125" valign="top" style="width:93.5pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"> </p>
</td>
<td width="125" valign="top" style="width:93.5pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"> </p>
</td>
</tr>
<tr>
<td width="120" valign="top" style="width:89.75pt;border-right:1pt solid windowtext;border-bottom:1pt solid windowtext;border-left:1pt solid windowtext;border-top:none;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif">$RE</p>
</td>
<td width="130" valign="top" style="width:97.25pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"> </p>
</td>
<td width="125" valign="top" style="width:93.5pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"> </p>
</td>
<td width="125" valign="top" style="width:93.5pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"> </p>
</td>
<td width="125" valign="top" style="width:93.5pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"> </p>
</td>
</tr>
<tr>
<td width="120" valign="top" style="width:89.75pt;border-right:1pt solid windowtext;border-bottom:1pt solid windowtext;border-left:1pt solid windowtext;border-top:none;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif">$NT</p>
</td>
<td width="130" valign="top" style="width:97.25pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif">4 options: </p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif">Target child (TCH), Parent (PAR), </p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif">Other Child (OCH), </p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif">Other Adult (OAD)</p>
</td>
<td width="125" valign="top" style="width:93.5pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif">5 options </p>
</td>
<td width="125" valign="top" style="width:93.5pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif">14 options</p>
</td>
<td width="125" valign="top" style="width:93.5pt;border-top:none;border-left:none;border-bottom:1pt solid windowtext;border-right:1pt solid windowtext;padding:0in 5.4pt">
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif">If adult selected for sub-code #1 (PAR or OAD), 4 options for
sub-code #4. </p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif"> </p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;line-height:normal;font-size:11pt;font-family:Calibri,sans-serif">If child selected for sub-code #1 (TCH or OCH), 3 options for
sub-code #4. </p>
</td>
</tr>
</tbody></table><br class="gmail-Apple-interchange-newline"></span></td></tr></tbody></table>Should N = 5? Or the total number of possibilities? Or...? <div><br></div><div>Additionally, if coders tend to be very reliable on the main code, sub-code #1, and sub-code #2, which are more straightforward and leave less room for interpretation, how can I go about calculating the reliability of sub-code #3 and sub-code #4 separately? </div><div><br></div><div>Thank you so much for your consideration! </div><div><br></div><div>Best, </div><div>Ari Nelson<br><div><br></div><div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">Ariadne Nelson, M.A.</div><div dir="ltr">Pronouns: she/her/hers<br>Doctoral Candidate<br>Applied Developmental and Educational Psychology<br>Lynch School of Education<br>Boston College</div></div></div></div></div></div></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups "chibolts" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:chibolts+unsubscribe@googlegroups.com">chibolts+unsubscribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/chibolts/CAF%3DFrmyauN49KG00E5deDN1dayBErMs0_CXWPAVjnGwxUi2Myw%40mail.gmail.com?utm_medium=email&utm_source=footer">https://groups.google.com/d/msgid/chibolts/CAF%3DFrmyauN49KG00E5deDN1dayBErMs0_CXWPAVjnGwxUi2Myw%40mail.gmail.com</a>.<br />