calculating reliability on main lines

[Leonid Spektor]

 Janet,

	RELY program has some peculiarities in the way it gives feedback. The "Done with file ..." messages give more information than is necessary and this causes the confusion. RELY creates only one output file. The message "Done ..." means that the file has been read, unlike in other CLAN programs where it means that the file by that name has been created. I have refined the messages that RELY displayed and in the new version it will be much more clear. I will install the new version by Monday.

The message 'CURRENT OUTPUT FILE "sample2.rely.cex" IS INCOMPLETE' is erroneously displays wrong file name. In your example it should say 'CURRENT OUTPUT FILE "sample.rely.cex" IS INCOMPLETE' instead. I have fixed this error as well. This message indicates that whatever file CLAN program has created is corrupted and therefor should be ignored. RELY has one limitation that would produce this kind of error message. If the tier names at a particular location in both input files do not match, then RELY displays this message and quits. RELY assumes that at least all speaker tier names will be the same and in the same order within both input files. The text on those tiers can differ and they can have different number of dependent tiers, but if the speakers themselves do not match, then there is a really big problem there and it needs to be fixed before RELY can continue. 

If you still have problems or questions about RELY please write to me directly and please attach all the input files that you use in your RELY program examples.

Leonid.





On Jan 22, 2011, at 12:04, Janet Bang wrote:

> Hi Dr. MacWhinney,
> 
> Thank you for your suggestion.  For reliability, we were specifically wanting to look at the level of the words per utterance. Actually, the RELY program was able to give us an output of comparisons on the main line, which is a big help in having much of the data ready, to then do the statistical calculations by hand.  Therefore, my question now has to do with automating this process, rather than automating computation. 
> 
> What we mean about the RELY program giving us an output of comparisons, is that the program was able to recognize the following situation:
> - if the file sample.cha had an utterance "mommy goes home" on line 16
> - and sample2.cha had the utterance "mommy goes back home" on line 18
> - the RELY program was able to recognize that these two utterances contained the same information and compared them appropriately in the output as being different. 
> 
> However, after testing out the RELY program on the main lines (I used the command: rely +t*MOT -t*CHI -t*%), I realized that the program was not be able to produce an output comparing these lines past line 60 on the original file. 
> 
> I've tested out the .cha files at various lengths, and this seems to be where the output stops, although I've set the limit of lines in the output as "no limit".
> 
> Furthermore, in the output window when I am running the RELY program on sample.cha and sample2.cha, this information shown:
> 
> CURRENT OUTPUT FILE "sample2.rely.cex" IS INCOMPLETE
> 
> This is a bit confusing to us because even with the shorter files where this program does not produce this note, it will say:
> 
> Done with file <sample.rely.cex> with errors reported in <sample.rely.cex>
> Done with file <sample2.rely.cex>
> 
> But, there is no file called <sample2.rely.cex> on the computer, even when it seems the program was able to run to completion.
> 
> Is there something that we are not considering in the command when we run this program? Or, is it that the RELY program was just not meant to compare the main lines, and this is why this may be happening?
> 
> 
> Thank you for your help,
> 
> Janet Bang
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On Fri, Jan 7, 2011 at 10:39 AM, Brian MacWhinney <macw at cmu.edu> wrote:
> Dear Janet,
>    Yes, we discussed this earlier.  If I remember correctly, we concluded that the shape of the main line is so complex that there was no well-defined way of computing reliability.  Would it be on the level of the word, or the phoneme, or the code or the overall sentence, or exactly what?  Perhaps you could create a small sample file in which you demonstrate how you think reliability should be computed and we could then take a look at that to see whether the procedure could be automated.
> 
> -- Brian MacWhinney
> 
> On Jan 7, 2011, at 1:01 AM, Janet wrote:
> 
> > Dear users of CHAT and CLAN,
> >
> > I am a graduate student at McGill University using CHAT and CLAN to
> > transcribe and analyze parent-child interactions.  I would like to
> > establish reliability of these transcriptions with other transcribers,
> > and was wondering if anyone has been able to use the RELY program to
> > do so? I noticed there was a posting to this question a couple of
> > years ago, and there wasn't a way for RELY to do so on the main
> > lines.  Any help would be much appreciated.
> >
> > Thank you,
> >
> > Janet Bang
> >
> > --
> > You received this message because you are subscribed to the Google Groups "chibolts" group.
> > To post to this group, send email to chibolts at googlegroups.com.
> > To unsubscribe from this group, send email to chibolts+unsubscribe at googlegroups.com.
> > For more options, visit this group at http://groups.google.com/group/chibolts?hl=en.
> >
> >
> 
> --
> You received this message because you are subscribed to the Google Groups "chibolts" group.
> To post to this group, send email to chibolts at googlegroups.com.
> To unsubscribe from this group, send email to chibolts+unsubscribe at googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/chibolts?hl=en.
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups "chibolts" group.
> To post to this group, send email to chibolts at googlegroups.com.
> To unsubscribe from this group, send email to chibolts+unsubscribe at googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/chibolts?hl=en.

-- 
You received this message because you are subscribed to the Google Groups "chibolts" group.
To post to this group, send email to chibolts at googlegroups.com.
To unsubscribe from this group, send email to chibolts+unsubscribe at googlegroups.com.
For more options, visit this group at http://groups.google.com/group/chibolts?hl=en.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/chibolts/attachments/20110122/5bff7b43/attachment.htm>