Stefan,<div><br></div><div>> Are these really MI scores? At least by default, the Sketch Engine calculates something</div><div>> different, which Adam calls a "salience score".</div><div><br></div><div>MI has never looked like a very good collocation statistic: it gives too much prominence to rare items. We used to use a variant on it but it didn't scale well so we changed a couple of years ago to a (scaled version of) Dice, as Dice scored best in James Curran's extensive evaluation (in relation to distributional thesauruses) in his PhD.</div>
<div><br></div><div>For bigrams (tho not for word sketches) we do offer MI as an option: go to any concordance and click on the 'collocation' button.</div><div><br></div><div>Stats used in word sketches are not so transparent because we also consider the frequency of the grammatical relation, but all is explained in the help pages, see </div>
<div><span class="Apple-style-span" style="font-family: Verdana, sans-serif; font-size: 12px; line-height: 16px; "><a class="attachment" href="http://trac.sketchengine.co.uk/attachment/wiki/SkE/DocsIndex/ske-stat.pdf?format=raw" title="Attachment SkE/DocsIndex: ske-stat.pdf" style="text-decoration: none; color: rgb(51, 113, 186); border-bottom-width: 1px; border-bottom-style: dotted; border-bottom-color: rgb(187, 187, 187); ">Statistics used in the Sketch Engine</a> at <span class="Apple-style-span" style="font-family: arial; font-size: small; line-height: normal; "><a href="http://trac.sketchengine.co.uk/wiki/SkE/DocsIndex">http://trac.sketchengine.co.uk/wiki/SkE/DocsIndex</a></span></span></div>
<div><br></div><div>I do think this question is much overrated and overresearched. (I've reviewed about a hundred papers on comparing stats and they're all inconclusive.) Several things matter more than the stat you choose: above all, how good your corpus is, in terms of size, composition and cleanliness. If you want better collocation lists, put some effort into finding/building/improving the corpus instead of fussing about stats. </div>
<div><br></div><div>Even if the corpus is held constant, grammatical analysis will help you more than choice of stat. Stefan Wermter and Udo Hahn show this clearly in "Collocation Extraction Based on Modifiability Statistics", COLING 2004, <a href="http://acl.ldc.upenn.edu/C/C04/C04-1141.pdf">http://acl.ldc.upenn.edu/C/C04/C04-1141.pdf</a>. If the gramatical analysis is right, you don't need any stats: you can list the good collocations by simply using raw frequency. The point of the stats was to rule out common grammar words like (in English) "a" "the" "is" that otherwise always turn up everywhere. But any grammatical analysis will also rule them out. This corresponds to requests from professional lexicographers at OUP, Inst of Dutch Lexicology, Slovene Lexical Database project and elsewhere: they want to see the (grammatically constrained) collocations listed according to raw frequency as well as 'salience' (eg logDice, see above). In the Sketch Engine, word sketches can be sorted according to raw frequency or logDice. (See options on word sketch page)</div>
<div><br></div><div>O<font class="Apple-style-span" face="arial, helvetica, sans-serif">f course, evaluating statistics is a nice neat task but as I receive another paper about it to review, I do sometimes think of Ralph Waldo Emerson's 'a foolish consistency is the hobgoblin of small minds.'</font></div>
<div><br></div><div>Adam<br><br><div class="gmail_quote">2009/9/29 Stefan Evert <span dir="ltr"><<a href="mailto:stefan.evert@uos.de">stefan.evert@uos.de</a>></span><br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im"><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Michael B. gave the MI formula from COLLOCATES as:<br>
<br>
MI = log2 ( ( N^(s-1) * f (x y) ) / ( f (x) * f (y) ) )<br>
</blockquote>
<br></div>
Not really. What Michael gave was (shown here in a more explicit notation)<br>
<br>
MI = log2 ( ( N^(s-1) * f (x1 x2 ... x_s) ) / ( f (x_1) * f (x_2) * ... * f(x_s) ) )<br>
<br>
which is the (AFAIC correct) MI equation for n-grams (where n = s), i.e. for a combination of s consecutive words.<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I use (<a href="http://corpus.byu.edu" target="_blank">http://corpus.byu.edu</a>):<br>
<br>
MI = log10 ( ( N * f (x,y) ) / ( f (x) * f (y) * S ) ) / log(2)<br>
(divide by log(2), since LOG in SQL Server is base 10)<br>
<br>
where N = corpus size and S = span size.<br>
</blockquote>
<br></div>
This is a reasonable approximation of MI scores for surface collocations with a span size of S, i.e. combinations of _two_ lexemes which co-occur within a distance of at most S words.<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Brett R. gives:<br>
<br>
MI = log2 ( ( N * f (x y) ) / ( f (x) * f (y) ) ) ( where is the span ?)<br>
</blockquote>
<br></div>
This is the MI score for adjacent bigrams, which is compatible with both formulas above: in Michael's version, you have to set s=2 (for a bigram), in Mark's version, you have a window size of S=1 (for a 0L/1R span).<div class="im">
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
This is apparently the same or quite similar to what is used for BNCweb.<br>
</blockquote>
<br></div>
Yes, we use the correct mathematical model for surface collocations, which puts much more strain on the SQL database, but is usually close to your approximation.<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
One other question, I guess, is why Sketch Engine gives scores that are 40-50% off what is going on with BNCweb and BYU-BNC. I'm not saying that one is wrong and the other is right, but it's a bit disconcerting that the scores are not more similar. Maybe everyone could "cough up" their formulas, and we could see what's going on.<br>
</blockquote>
<br></div>
Are these really MI scores? At least by default, the Sketch Engine calculates something different, which Adam calls a "salience score".<br>
<br>
Cheers,<br><font color="#888888">
Stefan</font><div><div></div><div class="h5"><br>
<br>
<br>
_______________________________________________<br>
Corpora mailing list<br>
<a href="mailto:Corpora@uib.no" target="_blank">Corpora@uib.no</a><br>
<a href="http://mailman.uib.no/listinfo/corpora" target="_blank">http://mailman.uib.no/listinfo/corpora</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br>================================================<br>Adam Kilgarriff <a href="http://www.kilgarriff.co.uk">http://www.kilgarriff.co.uk</a> <br>
Lexical Computing Ltd <a href="http://www.sketchengine.co.uk">http://www.sketchengine.co.uk</a><br>Lexicography MasterClass Ltd <a href="http://www.lexmasterclass.com">http://www.lexmasterclass.com</a><br>
Universities of Leeds and Sussex <a href="mailto:adam@lexmasterclass.com">adam@lexmasterclass.com</a><br>================================================<br>
</div>