[Corpora-List] QM analogy and grammatical incompleteness

[Rob Freeman]

On Sunday 18 December 2005 19:00, Dominic Widdows wrote:
> Dear Rob,
>
> > For instance, famously, you can perfectly describe the momentum or the
> > position of a particle, but not both at the same time. This is
> > Heisenburg's Uncertainty Principle.
>
> It is, and the Uncertainty Principle is perhaps the clearest formal
> expression we have of the homely truth "you can't know everything at
> once". A more classical version of the the same argument would follow
> from the observation that, if you had a machine that tried to run a
> Laplacian / deterministic model of the universe, its physical size and
> the speed of light would limit the amount of information it could
> synchronously process.

If the speed of light were infinite would this still result in an Uncertainty 
Principle? I'm wondering because I want to know exactly where the Uncertainty 
Principle begins. My own model is just inconsistent orderings of a set, say 
according to colour and size, where you know everything about both orderings 
at every moment, but only one ordering is possible at any given moment. Such 
orderings are fundamentally inconsistent so you get an uncertainty principle. 
Is this the same thing as an information lag due to a finite speed of light?

> > So it is not so much the fact of going from a continuous quality
> > to a discrete quality which is interesting, it is the necessary
> > incompleteness of description in terms of discrete qualities
> > abstracted from a distribution which is where I think we should
> > be focusing, in analogy with the Uncertainty Principle of physics.
>
> Is this similar to asking whether all such quantization is a "lossy"
> transformation? Is this what you mean by incompleteness?

I think so. Yes.

> > Dominic, I have only read the publically available chapter
> > of your book. You mention a "vector model" for quantum
> > mechanics. Do you have anything on the Web which talks
> > about that? I can only recall ever having met descriptions of
> > QM in terms of functions.
>
> The article at http://plato.stanford.edu/entries/qm/ looks like a good
> place to begin for QM and vectors.

Hmm, blushes :-) It is of course trivial that 3-space is a vector, so any 
explanation of the physical world is also a vector explanation.

Still, I was thinking more of a vector explanation which motivated the quantum 
element. I was interested in this because it strikes me that any quantum 
element in grammar, and particularly any kind of grammatical uncertainty 
principle, is crucially dependent on a vector representation.

It hadn't struck me that vector representation is crucial to QM. Newton's laws 
apply to vectors, but they are not QM.

I'll have to look at it more closely and see if QM can be predicted purely 
from the fact that physical laws can be expressed in terms of vectors.

> In broad strokes, the history of vectors and functional analysis became
> very closely linked in the 1840s and 1850s, partly through Hamilton's
> work on quaternions and the theory of analytic functions on 4-space.
> Functions over the real numbers form a vector space - you can add two
> functions together, and multiply any function by a scalar. As a result,
> mathematicians came to realize that Fourier analysis could be described
> in vectors - each of the functions sin(nx) and cos(nx) (for x a real
> number, n an integer) is a basis vector, and any piecewise smooth
> function can be expanded (uniquely) as a vector, using these functions
> as a basis. The Fourier series coefficients are thus interpreted as the
> coordinates of a vector in this basis. This vector space is clearly
> infinite-dimensional, because a Fourier series expansion can be
> infinitely long. (Note again that this means you will never work with
> complete information once you've quantized your functions.)

So quantization always means a loss of information, complete information about 
one quantization will always result in incomplete information about another 
quantization, resulting in an uncertainty principle?

We still don't have any postulates, but we already have an Uncertainty 
Principle?

Is this dependent on the fact that the Fourier coefficients are 
infinite-dimensional?

> Make your 
> functions complex-valued, and introduce a metric based on
> complex-conjugation, and you've got Hilbert spaces, around 1900 I
> think.
>
> In the 1930's, Paul Dirac, John von Neumann, and others, used this
> formulation of functional analysis as the basis for formal quantum
> theory, much of which boils down to the analysis of self-adjoint
> operators on Hilbert space. Each function is a state-vectors, can be
> normalized and operated on. The resulting operator algebra (group of
> linear transformations under composition) is non-commutative, and this
> how the formal theory accounts for the Uncertainty Principle - the
> lower bound on the uncertainty of two observables is given by the
> magnitude of the commutator of their self-adjoint matrices.

That is what I was looking for. I didn't know that. It struck me later dealing 
with language that you would get an uncertainty principle if you defined 
grammar in terms of different ways of ordering word associations, but I 
didn't know that people had made the connection in QM, motivating the 
Uncertainty Principle from first principles dealing with vector 
representations of physical qualities.

In fact the vector representations they deal with are representations of 
functions though, right, functions expressed as sums of Fourier coefficients? 
So it is the act of summing over functions (to specify new functions) which 
gives you QM, not just the fact of vector representation? Well, I suppose 
that _is_ vector representation (for functions) fair enough.

In point of fact I had forgotten that quantum wave functions are usually 
expressed as vectors of Fourier expansion coefficients. Comes of too much 
familiarity with Greek letters :-) The standard formulation of QM is actually 
a vector formulation.

> Clear as mud? ;)

I think I am good with that. It has explained something I wondered about. The 
key to the Uncertainty Principle is that you collect components together 
(quantize) in QM, while in Classical Mechanics you don't. Is that right?

So, look, Dominic, moving from QM back to language, doesn't it then follow 
that any instantaneous characterization of grammar, if it is defined 
fundamentally as a clustering (quantization) of distributions of word 
associations, will be necessarily incomplete, for the same reason that there 
is inevitably a loss of information when you quantize state-vectors.

Just to underline that, it seems like quite an important result: doesn't this 
then mean that all machine learning, HPSG, all attempts to comprehensively 
describe grammar as a single comptete and consistent set of generalizations, 
any consistent attempt to enumerate grammar at all, HMM's etc. which seek 
complete representations for the grammar of a language in terms of a single 
set of consistent classes, all of these are doomed to fail?

Doesn't this then mean that we will always be wasting our time trying to 
completely characterize grammar, that the most complete description of 
language will only ever be a corpus, that we must leave language in the form 
of data, a corpus, and only think of grammar as coming into existence at the 
time it is observed, by making grammatical generalizations relevant to each 
problem, as that problem occurs?

So Dominic, by this result isn't a large part of contemporary linguistics (and 
knowledge representation/ontology as it happens), tree-banks, taggers, what 
have you, most of what gets talked about on this list, in fact, clearly 
wasting its time, trying to enumerate the innumerable?

> > I agree completely with your message, but would only add
> > that while quantum analogies can be very informative for
> > lexis, where I think it really gets interesting is in
> > syntax, which responds very nicely to a kind of "quantum"
> > analysis in terms of generating new quantum qualities
> > (particles?), a new one for each new sentence.
>
> This may be partly because composition is so far modelled more robustly
> in syntax that it is in (parts of) semantics? Just trying to figure out
> what compositional rules apply a list of noun-noun compounds extracted
> from a corpus is very hard - and this is just combining 2 "particles"!
> Some of the most interesting structures that arise in QM involve
> entanglement, and I dare say that some of the structures in syntax are
> as rich in this "multiply composed, new particles / systems arise"
> property. I don't have the expertise to do any proper analysis here,
> though.

I think syntax is where it gets interesting firstly because while people have 
done quite a lot with distributed representations for words, LSA, vector 
information retrieval, SVD etc, they have tended to shy away from syntax. 
Perhaps this is because with syntax language can no longer be seen as a 
classification problem. All our tools for dealing with distributed 
representations, neural-networks, case-based reasoning, have been 
classification tools. With syntax for the first time you can't get away from 
an ad-hoc character.

So syntax is firstly interesting because it has not been done. But secondly I 
think syntax is interesting because the very ad-hoc nature of syntax treated 
this way gives us vast new powers.

Because you can cluster contexts on words or words on contexts the analysis 
can be seen interchangeably as the generation of new syntax or the generation 
of new meaning. New syntax generates new meaning, and because of the quantum 
aspect, because you might say of very lossyness of quantization, the number 
of new meanings which can be generated is now greater than the size of the 
language itself. (Because new grammatical/semantic qualities are defined as 
one or other way of collecting sentences together, and you can fold lots of 
new patterns out of the same set of sentences.)

The ability of a finite language to express almost infinite shades of meaning 
is explained.

It rather reminds me of Goedel incompleteness, which allows a formal system to 
contain and describe itself. I don't know if there is another parallel there.

Are you aware of any other work which looks at syntax from this point of view?

Best,

Rob