[Lingtyp] Greenbergian word order universals: confirmed after all

[Harald Hammarström]

Pada Sel, 7 Nov 2023 pukul 16.54 Guillaume Jacques <rgyalrongskad at gmail.com>
menulis:

> Dear Martin,
>
> As you say, the reliability of these studies hinges on the cognate coding,
>> which is done manually, by humans with their biases. I'm wondering if there
>> is a way to measure the degree to which different linguists agree or not
>> (by some kind of kappa statistic), and a way to identify or exclude
>> systematic biases (which are part of normal human behaviour).
>>
> This is not something that has been done systematically in the past, but
> indeed a measure of inter-annotator agreement should be done in future
> research on phylogenies (when large teams are involved).
>

What would such an inter-annotator agreement score tell you? If the
annotator are novices just looking at wordlists obviously their
cognate coding will not be of patricular interest (but for figures on
agreement, see Fodor & Rapai 2006). If they are experts on the family
in question, they will have read each other's stuff, and their
judgments will not be independent.

I think the real lurking factor of interest is the implicit
subgrouping any human has to use for cognate coding in large-ish
families.

all the best,
H

Fodor, István & J. Rapai. (2006) Une nouvelle objection à la méthode des
"rassemblances" de Joseph H. Greenberg, fondée sur une expérimentation
psycholinguistique (psychométrique).  Bulletin de la Société de
linguistique de Paris CI(1). 439-456.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/lingtyp/attachments/20231223/583d202c/attachment.htm>