Refining early Basque criteria (miau)

[Sean Crist]

On Fri, 14 Jan 2000, Lars Henrik Mathiesen wrote:

>> From: larryt at cogs.susx.ac.uk (Larry Trask)

>> Nor does the use of a computer in any way guarantee greater
>> objectivity. If I decide to reject words showing property P, then I
>> can do this by hand or with a computer program, and the results will
>> be the same. After all, the computer isn't going to inform me that
>> it doesn't like my criteria.

> Given total objectivity in the researcher, that is true. However, in
> many scientific communities it is recognized that most researchers
> will unconsciously bias their evaluation of experimental data to
> reject more of those instances that will not support their theory.
[...]
> Having the researcher set the rules for acceptance of data and the
> computer apply them, would make it much harder to get a bias in there.

Actually, I think there's a real danger here.  The mechanical way in which
a computer carries out a query can give us an unwarranted confidence in
the objectivity of the results.  The problem is this: the database upon
which you run your query was created by humans with the same potential for
bias that you point out.

Let me give an example here.  Suppose that I'm creating a dictionary
database of Old English, and I'm coding each word for the dates and
dialects where it is attested. Well, scholars have differences of opinion
regarding the date and dialect of a particular document.  In such cases, I
have to make a decision one way or the other; I have to consider the
arguments that everyone gives and make my own best judgment about how I'm
going to code this particular controversial case.

So suppose you're querying my database, and you say, "Give me all the
words attested in such-and-such a century and dialect with properties X,
Y, and Z".  The computer will indeed be entirely mechanical in selecting
the words which meet these critera; but I hope it's clear that the output
will unavoidably reflect the choices I made when building the database to
start with.

I'm not saying that we shouldn't use computers for this kind of purpose;
quite to the contrary, I think that we've just scratched the surface of
what computers can do for work in historical linguistics.  My point is
that researcher bias is just as possible when you're using computers as
when you're doing the same work with paper and pencil.  I reject the
criticism that a set of findings are any less trustworthy because they
were produced without the aid of a computer.  Computers merely allow you
to produce the same bias-prone results more quickly.

  \/ __ __    _\_     --Sean Crist  (kurisuto at unagi.cis.upenn.edu)
 ---  |  |    \ /     http://www.ling.upenn.edu/~kurisuto/
  _| ,| ,|   -----
  _| ,| ,|    [_]
   |  |  |    [_]