<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor=white lang=EN-US link=blue vlink=purple>
<div class=WordSection1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Hi Yuri,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>It is possible to use chi-squared when the samples are of
different sizes, but you need to know the reliable limits. Along with two statisticians
at Lancaster, I looked at this and wrote up an experiment to find the limits:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Rayson P., Berridge D. and Francis B. (2004). Extending the
Cochran rule for the comparison of word frequencies between corpora. In Volume
II of Purnelle G., Fairon C., Dister A. (eds.) Le poids des mots: Proceedings
of the 7th International Conference on Statistical analysis of textual data
(JADT 2004), Louvain-la-Neuve, Belgium, March 10-12, 2004, Presses universitaires
de Louvain, pp. 926 - 936.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><a
href="http://www.comp.lancs.ac.uk/computing/users/paul/publications/rbf04_jadt.pdf">http://www.comp.lancs.ac.uk/computing/users/paul/publications/rbf04_jadt.pdf</a><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Hope that helps.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Paul.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Dr. Paul Rayson<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Director of UCREL and Lecturer in Computer Science <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Computing Department, Infolab21, Lancaster University,
Lancaster, LA1 4WA, UK.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Web: <a href="http://www.comp.lancs.ac.uk/~paul/"><span
style='color:blue'>http://www.comp.lancs.ac.uk/~paul/</span></a><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Tel: +44 1524 510357 Fax: +44 1524 510492<o:p></o:p></span></p>
</div>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>
corpora-bounces@uib.no [mailto:corpora-bounces@uib.no] <b>On Behalf Of </b>Yuri
Tambovtsev<br>
<b>Sent:</b> 27 July 2010 11:08<br>
<b>To:</b> corpora@uib.no<br>
<b>Subject:</b> [Corpora-List] How to use Chi-square correctly<o:p></o:p></span></p>
</div>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<div>
<div>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>Dear
Corpora colleagues, some American linguists e.g.<o:p></o:p></span></p>
</div>
<div>
<p class=MsoNormal>Rob Malouf and Stefan Th. Gries University of
California, Santa Barbara wrote:<span style='font-size:10.0pt;font-family:"Arial","sans-serif"'><o:p></o:p></span></p>
</div>
<div>
<p class=MsoNormal>This is especially true when you're comparing really big
counts with really small counts, which is I think what Adam's rule of thumb is
meant to address. Once you've decided that applying the chi-square test
even makes sense, then questions like significance levels and Bonferroni
corrections come into play. Rob Malouf <br>
Department of Linguistics and Asian / Middle Eastern Languages San Diego State
University<br>
I wonder if all the linguists on the Corpora list are so advanced in math.
statistics. Being a simple linguist I did not understand anithing. I mean why
it is not possible to use Chi-square criterion when the samples are different
in size. On the contrary, I read in the books on Chi-square that it is also
possible to use it when the samples are not equal. However, I want to be on the
safe side, so I take the equal samples when comparing two transcribed texts. I
usually take a sample of 10000 speech sounds from longer texts. I take the
sentences from the long texts at random. When the sample is 10000 I stop. Is it
not possible to use the Chi-square in this way? I am sure the discussion of how
to use and how not to use the Chi-square criterion and other math. statistics
criteria in linguistics is very important. Looking forward to hearing for your
advice to <a href="mailto:yutamb@mail.ru">yutamb@mail.ru</a> Remain yours
sincerely Yuri Tambovtsev, Novosibirsk, Russia<span style='font-size:10.0pt;
font-family:"Arial","sans-serif"'><o:p></o:p></span></p>
</div>
</div>
</div>
</body>
</html>