two more questions about converting SALT files to CHAT format
Leonid Spektor
spektor at andrew.cmu.edu
Sun Nov 11 05:35:07 UTC 2012
Ying,
MLU will not count utterances ending with SALT symbols '>' and '^' (CHAT's symbols '+...' and '+/.'), if the next utterance of the same speaker starts with CHAT's Self-Completion symbol '+,' . Also, utterances will not be counted by MLU if they end with CHAT symbol '+.' . Perhaps SALTIN should convert SALT symbols '>' and '^' to some other CHAT symbols or we can add '+,' symbol on the next utterance of the same speaker. I guess it depends on whether SALT symbols '>' and '^' always mark utterances that should not be counted by MLU.
I have fixed the quotes conversion problem, but I would like to hold on releasing new version of CLAN until the MLU utterances counting issue is resolved.
Leonid.
On Nov 10, 2012, at 14:38 , Ying Lu wrote:
> Dear Dr. MacWhinney,
>
> Thank you so much for your reply (especially during the weekend:-)!!!
>
> It would be great that Leonid could revise SALTIN to successfully convert the quotation marks from SALT to CHAT format!
>
> For my second question regarding the > and ^ marks, SALT, by default, excludes the abandoned and interrupted utterances, as well as the unintelligible and nonverbal utterances, from the analysis set. But of course, one may uncheck any of these options to include the type(s) of utterances. Now the nonverbal utterances (marked by {}) and the utterances which contain unintelligible parts (marked by XXX) are excluded consistently by both SALT and CLAN in MLU calculation.
>
> Below is the "pretend file" I made to test the conversion of those marks. SALT ignores the utterances marked by >, ^, {}, XXXX, also words starting with *, while CLAN only ignores {}, XXXX, also words starting with *.
>
> In SALT:
> $ Child, Examiner
> + Language: English
> - 0:00
>
> C There is a boy whose name is Tom>
> C He like/3s fish/ing^
> C He went *to a lake.
> C He got XXXXX.
> C XXXX went back home.
> C {picks up book}.
>
> In CLAN:
> @Begin
> @Languages: eng
> @Participants: CHI Target_Child, INV Investigator
> @ID: eng|change_me_later|CHI|||||Target_Child|||
> @ID: eng|change_me_later|INV|||||Investigator|||
> @Time Start: 00:00
> @Comment: Language: English
> *CHI: there is a boy whose name is tom +...
> %mor: adv:loc|there v:cop|be&3S^aux|be&3S det|a n|boy
> rel|whose^pro:wh|whose^det:wh|whose v|name^n|name
> v:cop|be&3S^aux|be&3S n|tom +...
> *CHI: he likes fishing +/.
> %mor: pro:sub|he v|like-3S
> adj:part|fish-PROG^n:gerund|fish-PROG^part|fish-PROG +/.
> *CHI: he went 0to a lake .
> %mor: pro:sub|he v|go&PAST 0inf|to det|a n|lake .
> *CHI: he got xxx .
> %mor: pro:sub|he adj:part|get&PERF^part|get&PERF^v|get&PAST^aux|get&PAST
> .
> *CHI: xxx went back home .
> %mor: v|go&PAST v|back^n|back^adv:loc|back^adj|back n|home^adv:loc|home .
> *CHI: 0 .
> %act: picks up book .
> @End
>
> You are right that the > mark is converted to +... And yes, I will look for the instructions about how to create the postcodes! I just started to use CLAN, but I can see it is very powerful and helpful! I also have Spanish and Mandarin data to analyze. I would love to do the data exploration with CLAN!
>
> Thank you very much for your patience and time! I am really grateful for the help and information from you and Leonid!
>
> Sincerely,
> Ying
>
> On Sat, Nov 10, 2012 at 11:43 AM, Brian MacWhinney <macw at cmu.edu> wrote:
> Dear Ying Lu,
> About 10 months ago, we eliminated the use of the <quiet> ["] form in CHAT. It appears that Leonid needs to revise SALTIN to match the new format. The new format, described in section 7.9 of the manual, uses special Unicode marks for begin quote and end quote. Hopefully, Leonid can revise this part of SALTIN to correspond to the new format.
> Regarding utterance exclusion on the basis of the > and the ^ mark, I don't think that Brown (1973) specifies clearly that these should be excluded. The final > mark should be converted to +... Is that happening correctly? If not we can fix that. You don't provide any examples of the use of the ^ mark. I don't imagine anyone would really want to exclude all interrupted utterances, including those with just overlap. So, this would have to be defined more closely.
> If you want to fine-tune all of this, you can create postcodes for these various types and use those postcodes as the basis for exclusion. The problem here is that different people have different ideas about which utterances should or should not be included.
>
> -- Brian MacWhinney
>
>
> On Nov 9, 2012, at 7:03 PM, Ying Lu <yl5834 at gmail.com> wrote:
>
>> Dear Leonid,
>>
>> Thank you very much for your help! I am now able to convert files created in the newest version SALT to CHAT format!
>>
>> I have two more questions.
>>
>> One is about converting the words in quotation marks from SALT to CHAT format. It seems that CLAN will not count the words in quotations marks when running MOR. E.g.,
>>
>> In SALT:
>> C the little boy said[mv][IP] "quiet" [G].
>> In CLAN:
>> *CHI: the little boy said [% mv] [% ip] <quiet> ["] [% g] .
>> %mor: det|the adj|little n|boy v|say&PAST n|quote .
>>
>> The other is about how to exclude the abandoned utterances (marked by > in SALT) in CLAN. E.g.,
>> In SALT:
>> C (And they and they) ((and then)) they could/n't[IP]>
>> C ((And then)) the boy look/ed[mv] into the hole [G].
>> In SALT, the abandoned utterances will be excluded from the analysis set. But CLAN keeps them in the MLU calculation. The same happened to the interrupted utterances (marked by ^ in SALT).
>>
>> Thank you very much again!
>> Sincerely,
>> Ying
>>
>>
>>
>> On Monday, November 5, 2012 11:46:57 AM UTC-6, Spektor, Leonid: CMU wrote:
>>
>> SALT started adding Unicode identification characters at the beginning of transcript. I have changed CLAN in general and SALTIN in particular to recognize those characters. I will update CLAN by the end of today.
>>
>> Leonid.
>>
>>
>>
>> On Nov 5, 2012, at 11:51, Sophie Ambrose wrote:
>>
>>> I've recently begun running into this problem too and wasn't sure why the issue arose suddenly. The transcribing is being done in SALT elsewhere, so it's certainly possible the lab changed to the 2012 version recently too. I'm glad Leonid will be taking a look at this issue.
>>> Sophie
>>>
>>> On Monday, November 5, 2012 9:47:59 AM UTC-6, Spektor, Leonid: CMU wrote:
>>> Ying Lu,
>>>
>>> If the latest version of SALT does output some special characters, then it would be best for us to change SALTIN to recognize them. For that I need a copy of your "Tran1.slt" file. Please email it to me if you can.
>>>
>>> Leonid.
>>>
>>>
>>>
>>> On Nov 5, 2012, at 09:48, Ying Lu wrote:
>>>
>>>> Dear Leonid,
>>>>
>>>> Thank you very much for your reply!
>>>>
>>>> For problem (a), I understand the errors are due to the unrecognized initial characters of the tiers (e.g., $, C). But I don't know why that happened. I suspect there is a compatibility issue here, i.e., SALTIN can't recognize some symbols generated by the SALT 2012 student version. I will try other SALT versions later and see if that works.
>>>>
>>>> As for problem (b), I got a reply from Dr. MacWhinney last night and was able to get the comparable MLU. Thanks a lot again for directing me to use POST!!!
>>>>
>>>> Sincerely,
>>>> Ying Lu
>>>>
>>>>
>>>> On Mon, Nov 5, 2012 at 12:05 AM, Leonid Spektor <spe... at andrew.cmu.edu> wrote:
>>>> Ying Lu,
>>>>
>>>> (a)
>>>> The first error "** ERROR: unknown symbol <> found on 1 line." is probably because the first character on the line 1 is not '$' character. It might be some special control character that SALT or your text editor has inserted and CLAN doesn't recognize. The rest of the errors are due to first line not being recognized by SALTIN and as a result SALTIN can't identify speaker name code.
>>>>
>>>> I have created a test file from your example below and here is the result from SALTIN that I get:
>>>>
>>>> @Begin
>>>> @Languages: eng
>>>> @Participants: CHI Target_Child, INV Investigator
>>>> @ID: eng|change_me_later|CHI|||||Target_Child|||
>>>> @ID: eng|change_me_later|INV|||||Investigator|||
>>>> @Time Start: 00:00
>>>> @Comment: Language: English
>>>> *CHI: there is a boy whose name is tom .
>>>> *CHI: he likes swimming .
>>>> %tim: 00:05
>>>> @End
>>>>
>>>> It would be very helpful if you would send to me directly the "Tran1.slt" file that you used.
>>>>
>>>>
>>>> (b)
>>>> The difference in MLU results is due to %mor tier not being disambiguated by POST command. If you run POST command, then you will get this CHAT file:
>>>>
>>>> *CHI: the dog was [% ip] looking [% mv] in the beehive and 0at [% po] the mole [% u] .
>>>> %mor: det|the n|dog aux|be&PAST&13S part|look-PROG prep|in det|the n|beehive coord|and 0prep|at det|the n|mole .
>>>>
>>>> and this MLU result:
>>>>
>>>> MLU for Speaker: *CHI:
>>>> MLU (xxx, yyy and www are EXCLUDED from the utterance and morpheme counts):
>>>> Number of: utterances = 1, morphemes = 11
>>>> Ratio of morphemes over utterances = 11.000
>>>> Standard deviation = 0.000
>>>>
>>>>
>>>> Leonid.
>>>>
>>>>
>>>>
>>>> On Nov 4, 2012, at 20:33 , Ying Lu wrote:
>>>>
>>>>> Dear Chibolts group,
>>>>>
>>>>> I am trying to convert some SALT files and compare the MLU calculated by SALT with that by CLAN. I encounter some problems:
>>>>>
>>>>> (a) I am using SALT 2012 Student Version.
>>>>>
>>>>> The original SALT file is (I am using a short version sample here):
>>>>>
>>>>> $ Child, Examiner
>>>>> + Language: English
>>>>> - 0:00
>>>>>
>>>>> C There is a boy whose name is Tom.
>>>>> C He likes swimming.
>>>>>
>>>>> - 0:05
>>>>>
>>>>> But I got the following message when converting a SALT file into CHAT format using the command line "SALTIN Tran1.slt":
>>>>> > SALTIN Tran1.slt
>>>>> saltin Tran1.slt
>>>>> Sun Nov 04 18:39:34 2012
>>>>> saltin (07-Sep-2012)
>>>>> ****************************************
>>>>> From file <Tran1.slt>
>>>>> ** ERROR: unknown symbol <> found on 1 line.
>>>>> ** ERROR: unknown symbol <c> found on 5 line.
>>>>> ** ERROR: unknown symbol <c> found on 7 line.
>>>>> Output file <Tran1.sltin.cha>
>>>>>
>>>>> The output file looks like this:
>>>>>
>>>>> @Begin
>>>>> @Languages: eng
>>>>> @Time Start: 00:00
>>>>> @Time Start: there is a boy whose name is tom .he likes swimming .00:05
>>>>> @End
>>>>>
>>>>> I don't understand why the errors would occur.
>>>>>
>>>>> (b) Also, the MLU calculated by SALT is often different from that calculated by CLAN. For example,
>>>>> C The dog was look/ing in the beehive and *at the mole. (in SALT)
>>>>>
>>>>> When I convert this to CHAT format, and use the MOR and MLU commands, it showed:
>>>>> *CHI: The dog was [% ip] looking [% mv]
>>>>> in the beehive and 0at [% po] the mole [% u] .
>>>>> %mor: det|the v|dog^n|dog v:cop|be&PAST&13S^aux|be&PAST&13S
>>>>> adj:part|look-PROG^n:gerund|look-PROG^part|look-PROG
>>>>> prep|in^adv:loc|in^pfx|in det|the n|beehive coord|and 0prep|at
>>>>> det|the n|mole .
>>>>>
>>>>> The MLU by SALT is 11, but the Ratio of morphemes over utterances = 13 (Number of: utterances = 1, morphemes = 13) by CLAN.
>>>>>
>>>>> Apparently, the MLU result by SALT is what I want. But how could I get the same result in CLAN?
>>>>>
>>>>> Thank you very much for your attention! Look forward to your help!
>>>>> Sincerely,
>>>>> Ying Lu
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google Groups "chibolts" group.
>>>>> To post to this group, send email to chib... at googlegroups.com.
>>>>> To unsubscribe from this group, send email to chibolts+u... at googlegroups.com.
>>>>> To view this discussion on the web visit https://groups.google.com/d/msg/chibolts/-/z7qY4rboSlAJ.
>>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "chibolts" group.
>>>> To post to this group, send email to chib... at googlegroups.com.
>>>> To unsubscribe from this group, send email to chibolts+u... at googlegroups.com.
>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "chibolts" group.
>>>> To post to this group, send email to chib... at googlegroups.com.
>>>> To unsubscribe from this group, send email to chibolts+u... at googlegroups.com.
>>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>>
>>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "chibolts" group.
>>> To post to this group, send email to chib... at googlegroups.com.
>>> To unsubscribe from this group, send email to chibolts+u... at googlegroups.com.
>>> To view this discussion on the web visit https://groups.google.com/d/msg/chibolts/-/XeiXF8xIoBwJ.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups "chibolts" group.
>> To post to this group, send email to chibolts at googlegroups.com.
>> To unsubscribe from this group, send email to chibolts+unsubscribe at googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msg/chibolts/-/W2uyUInsDVcJ.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
>
> --
> You received this message because you are subscribed to the Google Groups "chibolts" group.
> To post to this group, send email to chibolts at googlegroups.com.
> To unsubscribe from this group, send email to chibolts+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "chibolts" group.
> To post to this group, send email to chibolts at googlegroups.com.
> To unsubscribe from this group, send email to chibolts+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
--
You received this message because you are subscribed to the Google Groups "chibolts" group.
To post to this group, send email to chibolts at googlegroups.com.
To unsubscribe from this group, send email to chibolts+unsubscribe at googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/chibolts/attachments/20121111/bc088463/attachment.htm>
More information about the Chibolts
mailing list