Call for Debate: Reproducibility in Typology

[Paolo Ramat]

Dear Frans,
thanks for your thought provoking lines.
I must confess that I have some problems in imagining what could be
'reproduced' in typology. I (think to) understand what Larry Hyman
(Aug.24th)  means by 'replication' of a research done by another set of
researchers with the same language sample. But 'replication' is not
'reproduction'. You may repeat/replicate a  chemical experiment in a lab and
obtain/reproduce the results of the previous experiment. But in the human
sciences this is not possible: even the best historian can't repeat
Waterloo's battle --and even in economics you may not repeat the NYSE
results of the week; you may 'simulate' them and, accordingly. make
forecasts for the stock exchange of the next week. No economist bothers
whether economics is a science or not: he/she simply does his/her job in
economical matters.
Why should typologists warry about the dilemma 'is linguistic typology a
science or not'? Moreover, the alternative is not, in my opinion, between
'science' and 'art': theere are different kinds of science along a gradient
of  'reproduceability' (as Edith has the word). Linguistic typology (a
science which is at the border of the human sciences) can hardly reproduce
claims, unless by 'reproduce' we mean 'make previsions' with more than
chance probability.
Take as a linguistic example the following phonetic evolution: PIE /p/ >
Engl. /f/ as in "father, foot, fish" etc. Finding a word like "five" one is
legitimate to guess that f- goes back to PIE p-. But this does not apply to
"finish, favour" etc. because these words were borrowed from French. The
non-reproducibility of the rule "PIE /p/ > Engl. /f/" is not detrimental to
the above 'Lautgesetz', which operates with more than chance probability. I
think the same holds in typological studies:  Greenberg's generalization
that with more than chance frequency SOV languages are postpositional cannot
be 'reproduced'/'replicated' via experiments. What typologists can do is to
guess that an SOV language will probably be postpositional and check whether
this guess applies to the language sample they have chosen.

Paolo Ramat


----- Original Message -----
From: "Frans Plank" <Frans.Plank at UNI-KONSTANZ.DE>
To: <LINGTYP at LISTSERV.LINGUISTLIST.ORG>
Sent: Tuesday, August 24, 2004 1:08 PM
Subject: Call for Debate: Reproducibility in Typology


> Re-doing Typology:  Call for Debate
>
>
> In good science one expects experiments to be reproducible:  results are
> only accepted if researchers other than those who designed and ran the
> original experiment and obtained results which are considered sufficiently
> interesting among the scientific community can independently repeat the
> same experiment and obtain the same results, over and over again.
>
> Scientific methods include not only experimentation but also surveys.  In
> the life sciences (like in many fields of biology), probably all
population
> and ecological sciences, sciences of the past (palaeontology, archaeology,
> etc.), and sociology surveys are used rather than experiments.
> Essentially, an experiment is where you manipulate conditions, and a
survey
> is where you select conditions.  Simulation is another standard procedure
> in some of the sciences, where experiments or surveys would be unviable.
> The replicability issue is equally important for surveys, experiments, and
> simulation.
>
> The purpose of this call for debate is to determine to what extent this
> principle of scientific method -- in order to be accepted, results must be
> reproducible -- is applicable to linguistic typology.
>
> For once, let's be catholic on what is typology.  It's linguistics in so
> far as it is about the human capacity for speaking and understanding, and
> its special remit is to observe, describe, and explain how particular
> manifestations of this capacity, as mentally representated and socially
> shared (grammars and lexicons, if you prefer), can and cannot vary across
> speech communities.
>
> The first real question then is whether typology is a science.  The
> alternative, to put it somewhat grossly, is that it is a form of art, like
> writing sonnets or interpreting them.   Most current practitioners of
> typology presumably opt for science, even when based in Arts or Humanities
> faculties rather than in Cognitive Science, and would like to see the
> results of their work evaluated as true or false rather than as beautiful
> or ugly -- or at any rate as true (yet unfalsified) or false (-ified)
first
> and as giving pleasure or pain to the senses only second.
>
> The next question is whether typologists perform experiments as part of
> their professional activities.  Regardless of what else typologists do in
> the way of observing, describing, and forming hypotheses, they arguably do
> experiments when they select subsets from a more comprehensive set (the
> known or knowable linguistic universe, since the typologist's results
> typically aim to be universal) and obtain descriptive generalisations from
> such a sample by induction.  Other typologists see themselves as doing
> surveys, selecting conditions rather than manipulating them.  Simulation
> has been done in typology, too, if less commonly than in other fields.
> What it means to be doing experiment-typology or survey-typology or
> simulation-typology is another question that remains to be clarified.
>
> Now, as to the primary issue which we would like to see debated, what
would
> it mean to reproduce a typological experiment or survey?   For example,
> would it be reproduced on the same or on a different sample?  How many
> typological experiments or surveys have actually been reproduced, one way
> or another?  (Probably not many:  identifying counterexamples to
> descriptive generalisations, which is being done a lot, is not
> reproduction.)  What are the obstacles hindering reproduction?  How to
> remove them?  Is it worth the effort?  (And the efforts required could be
> considerable, including for instance the obligation to make one's data and
> their interpretation available to fellow researchers, in publications or
on
> a web site.  Presumably, interpretations of data by typologists would
claim
> a different epistemological status than interpretations of sonnets by
> literary scholars:  in typology, data as well as interpretations ought to
> be reproducible.)  What follows if an experiment or survey is replicated
> but the results cannot be reproduced?  Is sampling the only part of the
> typologist's work where reproducibility is potentially an issue?
>
>
> LT invites contributions on the role of reproducibility in typology -- of
> whatever genre:  statements of principle, methodological discussions,
> reflections on your own experimental practice, actual attempts to
reproduce
> the results of others, reports on the difficulties encountered.
>
> Send to the editorial address of LT (for attached text files:
> frans.plank at uni-konstanz.de) by the end of October 2004.  A discussion
> section in LT 9-1 (2005) will be set aside for this purpose.  The
Editorial
> Board reserves the right to select submissions for publication.  Concise
> contributions are preferred, but longer papers on the subject are also
> welcome and will be reviewed as regular submissions.  In the first
instance
> you might also want to share your thoughts with the typological community
> on lingtyp at listserv.linguistlist.org
>
> As an alternative outlet, there is of course The Journal of Irreproducible
> Results (http://www.jir.com/).
>
>
> The Editorial Board of LT
>