transcript validation

[Elizabeth Prado]

I am working on transcribing children’s speech on the Indonesian island 
of Lombok, where the local language is Sasak. I’m working with 6 
transcribers, all of whom are native speakers of Sasak. 10% of every 
transcript is being re-transcribed by another transcriber for validation 
and we’re having difficulty getting high levels of agreement. I think 
there are two possible reasons for this: one is that Sasak is rarely 
written since all education from elementary school to university is 
couducted in Indonesian (the national language). The other is that there 
is significant dialect variation across the island. We are trying to 
give recordings of children to transcribers from the same dialect (same 
general area of the island) but this is difficult since dialect 
variation can occur from village to village.

The main purpose of the transcriptions is to validate a parent-report 
sentence complexity measure that we have developed to evaluate the 
language development of children whose mothers received micronutrient 
supplements during pregnancy.

I was wondering if anyone has transcribed any non-written languages and 
if you have any advice about how to increase agreement between 
transcribers. Even when we don’t count spelling differences as 
differences between the transcriptions, we’re still getting agreement 
<80%. Any advice would be appreciated!

-- 

*******************************************
Elizabeth Prado
Psychology Department
Fylde C Floor
Lancaster LA14YF
UK
Tel: 01524 592947
Website: http://www.psych.lancs.ac.uk/people/BethPrado.html