Alon,<div><br></div><div>the problem is - the differences are extremely likely to be statistically significant but that does not mean they are linguistically interesting - for the full explanation see </div><div><span class="Apple-style-span" style="font-family: 'Times New Roman'; font-size: medium; "><a href="http://kilgarriff.co.uk/Publications/2005-K-lineer.pdf">Language is never ever ever random.</a></span><span class="Apple-style-span" style="font-family: 'Times New Roman'; font-size: medium; "> </span><span class="Apple-style-span" style="font-family: 'Times New Roman'; font-size: medium; "><em>Corpus Linguistics and Linguistic Theory</em></span><span class="Apple-style-span" style="font-family: 'Times New Roman'; font-size: medium; "> </span><span class="Apple-style-span" style="font-family: 'Times New Roman'; font-size: medium; ">1 (2): 263-276.</span></div>
<div><font class="Apple-style-span" face="'Times New Roman'"><span class="Apple-style-span" style="font-size: medium;"><br></span></font></div><div><font class="Apple-style-span" face="'Times New Roman'"><span class="Apple-style-span" style="font-size: medium;">So you can't get an objective answer to the question 'is the difference noteworthy' (at least not until we have a far better theory of corpora) but there are some suggestions of the maths to support your analysis in <a href="http://kilgarriff.co.uk/Publications/2009-K-CLLiverpool-SimpleMaths.doc">Simple Maths for Keywords</a> (Proc. Corpus Linguistics, Liverpool 2009)</span></font></div>
<div><font class="Apple-style-span" face="'Times New Roman'"><span class="Apple-style-span" style="font-size: medium;"><br></span></font></div><div><font class="Apple-style-span" face="'Times New Roman'"><span class="Apple-style-span" style="font-size: medium;">Best</span></font></div>
<div><font class="Apple-style-span" face="'Times New Roman'"><span class="Apple-style-span" style="font-size: medium;"><br></span></font></div><div><font class="Apple-style-span" face="'Times New Roman'"><span class="Apple-style-span" style="font-size: medium;">Adam<br>
</span></font><br><div class="gmail_quote">On 13 October 2010 15:01, Alon Lischinsky <span dir="ltr"><<a href="mailto:alon.lischinsky@kultmed.umu.se">alon.lischinsky@kultmed.umu.se</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi.<br>
<br>
I am looking for help with a kind of statistical measure that has<br>
probably been described in the literature, but which I don't know how<br>
to call. I should point out that I'm relatively new to corpus studies,<br>
having a background in qualitative discourse studies, and am still<br>
coming to terms with some of the technical lexis.<br>
<br>
Simply put, I want to find out, given two terms that are seemingly<br>
synonymous but different in absolute frequency (say, "potato" and<br>
"spud"), which (lexical) terms have statistically significant<br>
differences in their collocation with either. I suppose I could simply<br>
look at the full list of collocates for each term ordered by t-score<br>
or MI and spot differences, but since one of the terms is much rarer<br>
and MI scores are affected by absolute frequency, I guess this might<br>
lead to quite a few artifacts.<br>
<br>
I don't know of any piece of software that can do that, so I would<br>
appreciate any pointers, or even suggestions as to how to go about<br>
doing it in R or any other statistical software (my programming skills<br>
aren't great, but I trust I could manage with a little guidance).<br>
<br>
Best,<br>
<br>
Alon Lischinsky<br>
<br>
_______________________________________________<br>
Corpora mailing list<br>
<a href="mailto:Corpora@uib.no">Corpora@uib.no</a><br>
<a href="http://mailman.uib.no/listinfo/corpora" target="_blank">http://mailman.uib.no/listinfo/corpora</a><br>
</blockquote></div><br><br clear="all"><br>-- <br>================================================<br>Adam Kilgarriff <a href="http://www.kilgarriff.co.uk">http://www.kilgarriff.co.uk</a> <br>
Lexical Computing Ltd <a href="http://www.sketchengine.co.uk">http://www.sketchengine.co.uk</a><br>Lexicography MasterClass Ltd <a href="http://www.lexmasterclass.com">http://www.lexmasterclass.com</a><br>
Universities of Leeds and Sussex <a href="mailto:adam@lexmasterclass.com">adam@lexmasterclass.com</a><br>================================================<br>
</div>