[Lingtyp] Areal and phylogenetic researcher biases

Sun Sep 29 13:50:13 UTC 2024

Dear Jürgen and all,

Thanks a lot for starting this important discussion. My thoughts about Jürgen's query were very much along the lines of Volker's reply, although his formulation was a lot better than mine would have been..

Probably just stating the obvious, but of course Volker's levels 2 and 3 of behavioural data each involve two processes that introduce further uncertainty into the analysis: the language data a researcher actually manages to record and the language descriptions that become influential enough to make it into typological samples. I'd say that that results in (at least) five layers at which typological data is 'behavioural' and each of these is shaped by a psychosocial context and (imagined) communities:

  1.
What speakers happen to say or do
  2.
What the researcher happens to record (i.e. what speakers happen to say or do when the researcher is observing)
  3.
What happens to make it into the descriptive analysis (and how)
  4.
What happens to make it into the typological sample
  5.
What happens to make it into the typological analysis (and how)

Obviously much of the methodological work in Linguistic Typology is exactly about constraining these levels of uncertainty, but I, for one, am skeptical about whether they can be eliminated in any meaningful way. The best we can hope for most of the time is to make the choices operating at each of these levels as explicit and motivated as possible (only to gloss over them for ultimate generalisations).

The best argument for why a typologist should at least once do a little bit of fieldwork is because it immediately confronts you with the reality that your recorded language is not the same as the language spoken in the community and that your description, even for the most basic phenomena, is a collection of editorial and authorial choices.

Best,
Stef

________________________________
From: Lingtyp <lingtyp-bounces at listserv.linguistlist.org> on behalf of Volker Gast via Lingtyp <lingtyp at listserv.linguistlist.org>
Sent: Sunday, September 29, 2024 10:41 AM
To: lingtyp at listserv.linguistlist.org <lingtyp at listserv.linguistlist.org>
Subject: Re: [Lingtyp] Areal and phylogenetic *researcher* biases

Dear Jürgen and others,

I think this is one of the major methodological problems of linguistic typology (which, if I remember correctly, has been discussed on this list before). There's no single 'correct' way of analysing a language. Two linguists working on the same language will often provide very different analyses, and both may be right in their own ways. It starts with phonology, where you have a lot of degrees of freedom in, for instance, minimizing or maximizing phoneme inventories (e.g. by [not] introducing phonological domains and features operating on these domains), and it gets worse in morphology, specifically if there is distributed exponence and other complexities of this type. At the level of syntax the impact of the specific theoretical background can be seen, for instance, in publications using the UD corpora. These corpora were annotated with a specific version of dependency grammar, I think essentially for pragmatic reasons (dependency grammar was very popular among computational linguists for a while). The theorerical assumptions of the annotation model obviously have an impact on the results (just think of the very old discussion of what a 'subject' is, represented as the 'nsubj' relation in the UD annotations).

For many languages we only have one description, and the linguist describing it comes from a specific background or 'school' (and these schools are often associated with particular areas and particular phylogenetic groupings, introducing further biases of the type you mention). Again, the effects are visible at the level of phonology already. For example, the Papuan language Idi could be described as having just three vowels, or as having nine vowels (perhaps even more), depending on your assumptions about phonotactics etc. (There's a published analysis of that language, by D. Schokkin, N. Evans, C. Döhler and me, but the analysis really reflects some kind of compromise between the authors, and it leaves a few non-trivial questions open.)

The specific linguist and their school or background is a source of statistical non-independence. Even relying on exactly one description per language, and having the data coded by several researchers, often leads to low inter-annotator agreement in my experience.

I think we need to be aware that typological data is behavioural data at three layers: (i) language is a behavioural activity, (ii) describing a language is a behavioural activity, and (iii) extracting information from descriptions is another behavioural activity. Variance occurs at all levels and is multiplied in the process from (i) to (iii).

Approximately determining the amount of variance of that type would be a major project. For instance, we could have five undocumented (unstandardized) languages described by five linguists each, using data from five different speakers per language. Many will think that this would be a waste of resources, given the number of (varieties) of languages that still await description.

What follows from all this, in my view, is that we need to be careful in applying statistical analyses "blindly". Linguistics is not a natural science. Given the large amount of inherent variance in typological data we linguists should remain in the driver's seat and use quantitative typological evidence as an assistance system, being aware of its limits and possibilities, rather than take a back seat and let the autopilot drive.

Best,
Volker

Am 28.09.2024 um 20:17 schrieb Juergen Bohnemeyer via Lingtyp:

Dear all – I’m wondering whether anybody has attempted to estimate the size of the following putative effect on descriptive and typological research:

Suppose there is a particular phenomenon in Language L, the known properties of which are equally compatible with an analysis in terms of construction types (comparative concepts) A and B.

Suppose furthermore that L belongs to a language family and/or linguistic area such that A has much more commonly been invoked in descriptions of languages of that family/area than B.

Then to the extent that a researcher attempting to adjudicate between A and B wrt. L (whether in a description of L, in a typological study, or in coding for an evolving typological database) is aware of the prevalence of A-coding/analyses for languages of the family/area in question, that might make them more likely to code/analyze L as exhibiting A as well.

So for example, a researcher who assumes languages of the family/area of L to be typically tenseless may be influenced by this assumption and as a result become (however slightly) more likely to treat L as tenseless as well. In contrast, if she assumes languages of the family/area of L to be typically tensed, that might make her ever so slightly more likely to analyze L also as tensed.

It seems to me that this is a cognitive bias related to, and possibly a case of, essentialism. (And just as in the case of (other forms of) essentialism, the actual cognitive causes/mechanisms of the bias may vary.)

But regardless, my question is, again, has anybody tried to guestimate to what extent the results of current typological studies may be warped by this kind of researcher bias? (Note that the bias may be affecting both authors of descriptive work and typologists using descriptive work as data, so there is a possible double-whammy effect.)

Thanks! – Juergen

Juergen Bohnemeyer (He/Him)
Professor, Department of Linguistics
University at Buffalo

Office: 642 Baldy Hall, UB North Campus
Mailing address: 609 Baldy Hall, Buffalo, NY 14260
Phone: (716) 645 0127
Fax: (716) 645 3825
Email: jb77 at buffalo.edu<mailto:jb77 at buffalo.edu>
Web: http://www.acsu.buffalo.edu/~jb77/

Office hours Tu/Th 3:30-4:30pm in 642 Baldy or via Zoom (Meeting ID 585 520 2411; Passcode Hoorheh)

There’s A Crack In Everything - That’s How The Light Gets In
(Leonard Cohen)

--

_______________________________________________
Lingtyp mailing list
Lingtyp at listserv.linguistlist.org<mailto:Lingtyp at listserv.linguistlist.org>
https://listserv.linguistlist.org/cgi-bin/mailman/listinfo/lingtyp

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/lingtyp/attachments/20240929/6a8b64bf/attachment.htm>