How new hypotheses grow

[ECOLING at aol.com]

In a message dated 8/24/99 12:28:35 AM, whiting at cc.helsinki.fi
is clearly offended by what I have written.
In part, I think the tone is self-evident for what it is,
and requires little response.

But the misunderstandings and misrepresentations I will try
to answer to some degree.

The major and underlying difference is that I am much more
consciously aware, or at least more concerned by the fact
that "methods" can yield wrong results, can be misapplied, etc. etc.

I have contributed a number of messages pointing out where
I think if we can make the data structures of our field of
historical linguistics more easily available, we may avoid
some of the "kook" noise, or at least shorten the lifetime
of fanciful creations which are without support.

I have also contributed many messages pointing out ways in
which we may be able to strengthen our tools, and I would prefer
that we in the field of historical linguistics do exactly that.
I believe the kinds of discussions in which people throw
theories (note! theories) of scientific method at each other,
neglecting the sociology of how new hypotheses are actually
discovered, are not profitable.  I believe it is actually fairly
well known that "philosophy of science" often makes errors
in disregarding the "history of science".  So I believe here also.

***

Here a couple of the kinds of challenges I favor.
I am sure experts in particular language families can
devise others better than these.

One good challenge
would be for historical linguists to figure out what kinds
of data pattern studies would serve to bring forth evidence
that could establish an Indo-European family based ONLY
on Albanian and some one other IE language. (Or which could
place Albanian in the IE family tree, where Ringe's team
said their tools could not do it.)  In other words,
take instances in which languages are known to be related,
and ask how could we have known that given much less data
then we in fact do have; what methods of comparison
and reconstruction are robust against such reductions of data sets?

Another good challenge:
A statement attributed to Ives Goddard some years ago will
also illustrate a good domain for posing more challenging
problems to improve our methods.  In substance it was:
[Wiyot and Yurok are clearly related to Algonquian,
but our traditional Comparative Method cannot establish that.]
It is many years ago now, and I am sure someone else
could report this better.

The question arises then, why is it that the best historical
linguists of American Indian languages were quite thoroughly
convinced AT SOME POINT that these languages were related,
yet the standard kind of evidence was somehow lacking?
What was it that they found so thoroughly convincing?
That is crucial, because it just might possibly suggest the
development of new tools which can penetrate deeper.
Experts' intuitions have a wonderful way of often,
not always, leading to demonstrable claims.  Intuition is not
formless, it can be studied and analyzed (after which we of
course no longer call it intuition, and conveniently forget
in many cases where it came from).  Nor are such successful
intuitions MERELY the tiny fraction of "guesses" with no
foundation, which happened by chance to turn out true.
In the most valuable cases, what we discover is a new kind
of evidence or way of handling data so that it becomes evidence.

Also, and equally important, what was it that the Amerindian
specialists felt at that earlier time point was not measuring
up to needed standards in the applicability
of the traditional comparative method to the data (or facts)?
If we discover that that kind of evidence, while of course
our standard and most convenient kind, is not absolutely
necessary to a convincing case, then we will have opened
new paths to gathering evidence in the future.
(Saying this does NOT imply I want to "relax" any standards
whatsoever, or be "lax" in any way whatsoever, it is simply
part of the process of discovering EMPIRICALLY what kinds
of data sets and tools are likely to yield arguments which
in the longer run lead to more evidence and to proofs.
Just like the tools we now use and TAKE FOR GRANTED,
except that while a tool is being developed it is not taken for
granted, and of course should not be.)

Well, there are two of the challenges I think would improve
our tools, if these challenges can be met well.

***

Mr Whiting clearly does NOT understand what I have written.
He apparently mistakes my attempts to counter what I regard
as the illegitimate parts of critiques as attempts to defend
claims which I have no wish to defend and have not defended.
(Ruhlen, see quotation below.)

Whiting also attributes to me, in magisterial tones, a
"spectacular lack of knowledge of what a hypothesis is."

Quite on the contrary, I know full well what a hypothesis is,
and I have absolutely no wish to support any kook claims,
which Mr. Whiting at least by innuendo seems to attribute to me.

BUT...
I HAVE ALSO grown up in a family with two parents
who were professional researchers, I am one myself,
and I learned very early how the supposed scientific method
can be manipulated to give the appearance of certainty where
there is none, EITHER to "prove" or to "disprove" a hypothesis.
I have learned how often people substitute "Straw man" hypotheses
which they can easily disprove, instead of the actual hypothesis
someone else really was proposing, which they cannot so
easily disprove.  I have learned to not consider that honest.
In short, I simply do not assume that those
who use the scientific method are completely unbiased,
perfect judges, or somehow gods above the rest of humanity.
That in no way means that I discount the importance of the scientific
method.  The latter does not in any way follow from the former,
and anyone who thinks so has not understood the balanced
sophistication of the reasoning I have presented.
I do not advocate extremes.  The fact that someone makes a
mistake on one item in no way leads me logically to conclude
that they are wrong on everything else as well.

If (as everyone now acknowledges,
it is a common saying) we all know that statistics can lie,
then why should the requirements of our traditional
"comparative method" be beyond question?
Particular statistical measures are known to be appropriate
only to problems of particular kinds of structure.
The same must almost certainly be true in a sense of our
traditional comparative method, that it works well in some
situations, poorly in others, and gives the wrong result
in yet other situations.  (I think a workbook in historical
linguistics which I used years ago in teaching actually
said explicitly that if the rules were followed correctly
with a particular data set involving languages of China,
that the answer would be a wrong answer.)

***

Here is merely one of many examples where Mr. Whiting
does not understand what I wrote, or chooses to argue
against a "Straw Man" instead of against what I actually wrote:

I wrote:

>As I have understood Joseph Greenberg's clearer and more cogent
>statements, his own work actually does NOT propose to prove any such
>conclusion.  It is rather an ASSUMPTION that all languages are or might
>be related (i.e. we are not to exclude that).

[that was to contrast with Ruhlen, who I believe is less explicit about that.
A small digression to clarify Greenberg's method.
All Greenberg's work does is
demonstrate NON-relatedness, not relatedness.
Or rather, his method seeks the greatest SEPARATIONS,
language groupings whose relationship (there by assumption not by proof)
is the MOST DISTANT.
I know this statement may seem paradoxical since Greenberg
is famous for claiming that all of Amerind except Athabaskan
and Eskimo-Aleut is one large family.  What that really means
is that Greenberg's method cannot establish any great cleavage
among the remaining Amerind languages, they seem to be a
large number of language families with overlapping similarities
without evidence dividing them into a very few super-super-families.
Whether that reflects chance similarities or true genetic relations
is beyond Greenberg's method.  He simply assumes everything
is related.
We can use his data collections (of course with
improvements and corrections wherever possible)
whether or not we agree with that assumption.]

Later I wrote:

>(I am much less familiar with Ruhlen than with Greenberg.)

to which Whiting remarked:

>Then you should perhaps familiarize yourself with his methodology
>before you try to defend it.

I did not defend Ruhlen's methodology in any terms having to do
with proving a hypothesis.
This is a clear misunderstanding or misrepresentation.
It is an example of what I call "Straw Man" reasoning.
If I HAD tried to defend Ruhlen's methodology  by claiming
Ruhlen was doing any kind of legitimate hypothesis testing
my logic would be easy to defeat, because Ruhlen has not done so.
But Whiting today treats me AS IF I had done so (that is the Straw Man),
and defeats that instead of arguing against what I actually did say.

Which does not address IN ANY WAY what I was actually saying
in this part of the discussion.

I had answered a comment by Larry Trask:

>>This fundamental failure to understand proper methodology is
>>enough to render Ruhlen's work vacuous,

as follows:

>Not so, since Ruhlen can be treated as involved in hypothesis
>FORMATION not hypothesis testing.

to which Whiting remarks:

>On the contrary, Ruhlen is not formulating a hypothesis (we all
>should know how to do this by now:  one starts with the data and
>then develops a hypothesis

sure, so far, that is what we do, and in the case of
Ket-Athabaskan (see below), that is I believe what Ruhlen did also.

continuing:
[a hypothesis]
>that accounts for the data [all the data]).

No, that last part is not anywhere near the early stages of
hypothesis formation, and many hypotheses which are
considered by the majority of professionals in a field to be
strongly confirmed do not account for ALL of the data.
In fact, perhaps no hypothesis EVER accounts for ALL of the data.
Attempting to insist on that too early in the process puts
a brake on all new discovery, by condemning the very process
which has been known, throughout the history of science,
to yield new discoveries...

Namely, the WILLINGNESS TO SUSPEND JUDGEMENT
FOR A WHILE, TO WORK WITH HYPOTHESES WHICH
ARE NOT YET STRONGLY CONFIRMED.

This is ONE of the two major illnesses of the wars in modern
historical linguistics, the eagerness to condemn tentative hypotheses
too early.  The other major illness is just the opposite, of course,
that is, the eagerness of some people to propose some
supposedly earth-shaking new discovery,
even if they really have not much to suggest it is worth pursuing,
simply because they are wannabe "discoverers".

Of course, we cannot "know" in advance how
particular hypotheses will turn out when we DO figure out how
to test them.
Often, we do not even know exactly how to state a hypothesis
formally at first (and therefore we do not know what its negative,
its corresponding null hypothesis, would be).

By BREAKING the continuum of growth of new hypotheses,
known in the history of science, we hamstring the development
of new knowledge.

Here is I think a paragraph from Mr. Whiting
which illustrates the inability of many linguists
to grant that "hypothesis" is an appropriate term
for ideas through a very broad range,
from shortly after initial glimmers,
through initial formal proposal (with as yet no test),
through until repeated tests have been passed
and we have a confirmed hypothesis.

>If Ruhlen can be considered to be involved in hypothesis
>FORMATION not hypothesis testing, then the hypothesis being
>formed is simply a hypothesis according to definition 4 above
>("a mere guess or assumption"), not a hypothesis according to
>definition 1 (a scientific hypothesis).

Of course, there is a difference between those two,
but there is also a continuum linking them.
Whiting continues in that same paragraph:

>Collecting data is not hypothesis formation;
>it is a prerequisite to hypothesis formation.

Whiting seems no longer willing to grant that anything prior to
his favored use of "hypothesis" could also be a "hypothesis".
Because the collecting of data is also a POSTREQUISITE
to most hypothesis formation, because hypotheses are formed
at VERY EARLY STAGES of most successful discoveries
or investigations.

>And the data on which a hypothesis is based cannot be
>used to test the hypothesis because that is simply circular.

That is actually done all the time.
People form hypotheses based on a set of data,
then apply a statistical test to the set of data, including in it
the same set of data from which they derived their hypothesis,
though hopefully in better practice at least an expanded set of data
in those cases where it is possible to obtain more data.

That is not sufficient of course,
but it is a universal procedure,
and it would be absurd to suggest otherwise.

Should a presentation of evidence for the Indo-European
language family systematically exclude all of the most convincing
data, the data which originally led to the tentative hypothesis?
Of course not.

In another paragraph, Whiting writes:

>It is true that many (most?) scientific hypotheses start as
>guesses.

I'm happy with that sentence.

>But not every guess is a scientific hypothesis.

I would have thought NONE of them are, the way he was
distinguishing "guess" from "hypothesis" earlier.

>While anything is possible, not everything is probable.
>Basically, no evidence, no scientific hypothesis.

But the presence of evidence does not guarantee a "scientific
hypothesis", presumably a term referring to a hypothesis
which is being subjected to scientific testing and is at least
quite a distance along the path of confirmation, if not confirmed.
Initial glimmers and guesses are ALSO based on evidence.
Data does not cease to be "evidence" merely because a hypothesis
referring to that data is not yet securely confirmed.
Not all "evidence" turns out to be valid evidence for what it was being
used as "evidence" for.  But it was still used as "evidence".
(basically a "showing" used in arguing in favor of something)

***

Concerning "Lookalikes"
I responded to Trask's:

>>and quite apart from his failure to realize that lookalikes do
>>not constitute evidence of any kind.

by saying:

>Disagree flatly, unless defined circularly so that "lookalikes"
>means more than it says, namely so that it means "lookalikes
>which are known to be unrelated as cognates".

>If it actually means "items which look alike in sound and
>meaning", then of course such comparisons DO constitute
>PRELIMINARY evidence. Any such preliminary evidence can be
>discounted by showing that the resemblances are secondary and
>late, or that they manifest a type of sound symbolism, or in
>other ways.

Whiting writes:

>To the extent that lookalikes are a fact and facts can be used as
>evidence, this is true.

[as in the formation of the initial hypothesis of the Indo-European
language family, which was first a guess, then a tentative
hypothesis (no longer merely a guess), and gradually became
more and more strongly confirmed...]

Whiting continues:

>But the question is and remains, what
>are they evidence of.  The null hypothesis is that they are
>evidence that similar sounding words with similar meanings occur
>in the world's languages by pure chance.

Wow, this is strange.
"The" [unique] null hypothesis?

No, that is NOT the null hypothesis to a claim of particular
lookalikes being real cognates.  That would rather be the
null hypothesis to counter a hypothesis that similar sounding
words with similar meanings do not (ever) occur in the
world's languages by pure chance.

Rather, the null hypothesis corresponding to a hypothesis
that a PARTICULAR set of look-alikes in the Indo-European languages
were evidence that they constituted a family, would be that those
particular similarities (of the look-alikes) arose from some other cause,
(including possibly chance resemblances or sound symbolism but
not limited to those, since this is merely the null hypothesis),
and in particular did not arise from descent from a parent language.

In his statement of the scientific method, Whiting
says ("3" is formulation of a hypothesis to account for the data,
1 and 2 are identification of a problem and collection of relevant data):

>But step 3 should be preceded by 1 and 2 in whatever order
>they may come.  Formulating a hypothesis and then
>collecting data to support it is not part of the scientific method.

Unless one uses words in such a way that ANY basis for formulating
a hypothesis constitutes either (1) or (2) by definition, then this
is not a valid description of how science proceeds.  The basis for
formulating a hypothesis is simply outside the scientific method,
a hypothesis can come from anywhere at all.  It is how one handles
the careful accumulation of data afterwards that makes for valid
science vs. sheer speculation.

***

Mr. Whiting really does seem to want to avoid discussion of
the possibility of improving the tools of historical linguistics.
He says:

>Many people ... point to the inadequacy of the tools
> used in historical linguistics.
Although to me this sounds like a version of "the
poor workman blames his tools",

[Notice that I at least phrased it just the reverse of that,
not blaming the tools, but looking to ways to strengthen them
and to add new tools, so we can perhaps address some range
of problems which our current crop of traditional comparativists
say are beyond possibility.  It is only reasonable to assume
that we can almost always make SOME progress of this kind,
in almost any field...]

Whiting continues:

>the primary problem of historical
>linguistics is not the inadequacy of the tools, but a lack of
>hard data to use the tools on.

I can even agree with the above!

But not with the following:

>When the data is available, the
>tools to investigate it will arise.

Because what his previous comment meant was that with
more data, the EXISTING tools would be adequate.
(Since there would be no impetus then to develop newer and
more powerful tools, his last statement is counter-indicated.)

***

Since Mr. Whiting illegitimately to tag me with being
a defender of Ruhlen's methodology (the more easily
to attack my arguements?), when I had NOT defended
Ruhlen's methodology as concerns hypothesis testing,
let me now specify the one place where I am most willing
to give up some of my time to read what Ruhlen has written.

Ruhlen has suggested that there may be a relation between
Ket (Yeniseian, in Siberia) and Athabaskan.
Is this worth spending any time on?
(That is the ONLY question I would ask at such an early stage
of any hypothesis, NEVER "is it proven?".)

Well, given that almost everyone agrees the Amerindians
of various groups are related to peoples of NE Asia,
and given that the Athabaskans are most likely a relatively
recent set of migrants from Asia, followed later only by
Eskimo-Aleuts,
and crucially given that there has not been much work on
such possible links (I think not, anyhow),
it is not impossible, and quite plausible, that there might be
some such relation.

My first question would be whether look-alikes are indeed
significantly easier to find between Athabaskan and Ket
than between Athabaskan and other languages of NE Asia.

Perhaps Greenberg's focusing on the lateness of
Eskimo-Aleut and Athabaskan migration to the Americas,
as compared with other Amerindian, has posed a problem
in such a way that one of the small number of people willing
to look for distant language relationships might happen
to see enough look-alikes to get the beginning glimmers
of a hypothesis.

That's how much of real science works, starting with
glimmers.
I have absolutely no idea whether this one will pan out
or not.  It is inherently conceivable enough,
and Ruhlen has no vested interest specifically
in Ket as opposed to any other language of NE Asia,
that if he judges Athabaskan to be more similar to Ket
than to other languages of that area,
it warrants listening.

New hypotheses must come from somewhere,
and they often must break the framework of our
established thinking.

What makes me saddest is that too many of those
who have taken on the responsibility of carrying our
society's knowledge in historical linguistics seem to
forget our roots in simple observation, and that those
roots do not go away as we add sophistication to
our observation
and more and more tools for the testing of hypotheses.

Remember the saying:
"Science = self-conscious common sense"

Sincerely yours,
Lloyd Anderson

***

For me, "Straw Man" has a very negative connotation, I learned
the term in the context of argumentation in which the debater
is not actually debating what was really proposed, but something
else, the more easily to appear to defeat it.
In a later part of his message, Whiting says:

>Discrediting STRAW MAN claims (= null hypothesis)
>is part of the method,

in which he appears to be treating Straw Man claims as
equal to null hypotheses.  Quite on the contrary, in the usage
I know, "straw man" claims are so named precisely because they
are NOT the real man, they are NOT what anyone was claiming,
and it is the null hypothesis OPPOSITE of such a straw-man
claim which the debater is trying to make seem more reasonable.