<div dir="ltr">The mutual information score that lexicographers use is a close relative of the mathematical notion of mutual information between two random variables. Peter Turney and others have been careful to reflect this distinction by using the term 'pointwise mutual information' (PMI) for the lexicographer's version and MI for the other. Technically, MI is the sum over all cells of a two dimensional matrix of the PMI. This means that you can begin to think of PMI as something like "the contribution of a particular pair of words to MI". And lexicographers have had fair success interpreting it this way. The mathematicians tend to look askance at PMI, because of concerns like "the PMI for a pair of words can in principle be negative even when the MI summed over all words is positive. What (the hell) does that mean?"<div>
<br></div><div style>MI is a central notion of information theory, and backed by many useful mathematical results. For the task of measuring word association, the mathematical advantages</div><div style>of MI do not really translate into a preference for using PMI rather than some other measure of association. If it works for you, that's OK. You don't get much extra from the connection to the mathematics.</div>
<div style><br></div><div style>Once you move to three or more terms, things get even more complex. The generalizations of MI to three or more terms are confusing in themselves, just because interactions between three or more variables are much more complicated than interactions between just two. The generalizations of PMI would be at least as messy, possibly worse, so it is no surprise that mathematical support for such generalizations is missing. </div>
<div style><br></div><div style><br></div><div style><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, May 14, 2013 at 10:14 AM, Mike Scott <span dir="ltr"><<a href="mailto:mike@lexically.net" target="_blank">mike@lexically.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
I have had a query about MI (or any other similar statistic)
involving more than two elements:<br>
<br>
<span style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:start;font-style:normal;display:inline!important;font-weight:normal;float:none;line-height:normal;color:rgb(34,34,34);text-transform:none;font-size:13px;white-space:normal;font-family:Arial,Helvetica,sans-serif;word-spacing:0px">"I don't know how
to calculate the Mutual Information (MI) for these 4-word lexical
bundles, it seems I can only find the MI score for 2-word
collocations."</span><br>
<br>
Can anyone advise please?<br>
<br>
Cheers -- Mike<span class="HOEnZb"><font color="#888888"><br>
<br>
<pre cols="72">--
Mike Scott
***
If you publish research which uses WordSmith, do let me know so I can include it at
<a href="http://www.lexically.net/wordsmith/corpus_linguistics_links/papers_using_wordsmith.htm" target="_blank">http://www.lexically.net/wordsmith/corpus_linguistics_links/papers_using_wordsmith.htm</a>
***
University of Aston and Lexical Analysis Software Ltd.
<a href="mailto:mike.scott@aston.ac.uk" target="_blank">mike.scott@aston.ac.uk</a>
<a href="http://www.lexically.net" target="_blank">www.lexically.net</a>
</pre>
</font></span></div>
<br>_______________________________________________<br>
UNSUBSCRIBE from this page: <a href="http://mailman.uib.no/options/corpora" target="_blank">http://mailman.uib.no/options/corpora</a><br>
Corpora mailing list<br>
<a href="mailto:Corpora@uib.no">Corpora@uib.no</a><br>
<a href="http://mailman.uib.no/listinfo/corpora" target="_blank">http://mailman.uib.no/listinfo/corpora</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr">Chris Brew</div>
</div>