transcript validation

[Matthew Saxton]

Dear Elizabeth,

 

Inter-rater reliability is vital, of course. But intra-rater reliability
comes first. I wonder how consistent your transcribers are being from
one transcript to the next? A simple way to check for this is to have
your researchers transcribe the same recording on two occasions - with a
decent interval between to dilute the effects of memory.

 

If there is still a problem, perhaps your transcribers could be given
some form of ear-training to enhance their ability to analyse speech in
purely phonological terms - in effect, reducing the (lexical) bias
introduced by hearing what you expect to hear. Even if you don't require
a high degree of phonological detail, this kind of focus might be
helpful in overriding the dialect effects you talk about.

 

Regards,

 

Matthew.

 

 

 

 

 

*********************************************************************

Matthew Saxton MA, MSc, DPhil

School of Psychology and Human Development,

Institute of Education,

25 Woburn Square,

London,

WC1H 0AA.

U.K.

 

Tel: +44 (0) 20 7612 6509

Fax: +44 (0) 20 7612 6304

 

http://ioewebserver.ioe.ac.uk/ioe/cms/get.asp?cid=4578&4578_0=10248

www.ioe.ac.uk

-----Original Message-----
From: info-childes at mail.talkbank.org
[mailto:info-childes at mail.talkbank.org] On Behalf Of Elizabeth Prado
Sent: 03 September 2007 09:16
To: info-childes at mail.talkbank.org
Subject: transcript validation

 

I am working on transcribing children's speech on the Indonesian island 

of Lombok, where the local language is Sasak. I'm working with 6 

transcribers, all of whom are native speakers of Sasak. 10% of every 

transcript is being re-transcribed by another transcriber for validation


and we're having difficulty getting high levels of agreement. I think 

there are two possible reasons for this: one is that Sasak is rarely 

written since all education from elementary school to university is 

couducted in Indonesian (the national language). The other is that there


is significant dialect variation across the island. We are trying to 

give recordings of children to transcribers from the same dialect (same 

general area of the island) but this is difficult since dialect 

variation can occur from village to village.

 

The main purpose of the transcriptions is to validate a parent-report 

sentence complexity measure that we have developed to evaluate the 

language development of children whose mothers received micronutrient 

supplements during pregnancy.

 

I was wondering if anyone has transcribed any non-written languages and 

if you have any advice about how to increase agreement between 

transcribers. Even when we don't count spelling differences as 

differences between the transcriptions, we're still getting agreement 

<80%. Any advice would be appreciated!

 

-- 

 

*******************************************

Elizabeth Prado

Psychology Department

Fylde C Floor

Lancaster LA14YF

UK

Tel: 01524 592947

Website: http://www.psych.lancs.ac.uk/people/BethPrado.html

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/info-childes/attachments/20070904/f3e2356a/attachment.htm>