Testing the testing tool itself

[ECOLING at aol.com]

In a message dated 3/26/99 8:17:16 PM, John McLaughlin wrote:

>The computer program simply gives us a feel for how close
>lexical similarity should be before we get excited enough to do the other
>comparisons.

Nice phrasing, appropriate.

I will unfortunately have no time to work on these questions in any detail
until summer.  But the ability to use actual languages, gaps and all,
without assuming we know anything about actual sound changes in advance,
is crucial for any proposed computer program.

My point is that we NEED TO TEST the purported test!
That is, the computer estimate of
random similarities needs to be structured so that it CAN
be applied to actual data of actual languages, WITHOUT
assuming sound changes or anything else in advance.
Then we can see to what extent we like its conclusions
on cases where the deep work of reconstruction HAS
been done, and if we don't, then we have no business
applying it to cases where the deep work has not been done.

The purported tool (testing for random similarities)
cannot be taken as even minimally valid unless it is itself
continually TESTED and improved.

Seems elementary to me.  Have I missed something?

Lloyd Anderson