<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix"> Good question, Jack!<br>
<br>
You're missing some information here. As you know,
statistical significance tells you whether your sample is large
enough that the effect you observe is probably not due to a bad
sample.<br>
<br>
<a class="moz-txt-link-freetext" href="http://grieve-smith.com/blog/2014/01/how-big-a-sample-do-you-need/">http://grieve-smith.com/blog/2014/01/how-big-a-sample-do-you-need/</a><br>
<br>
So you've figured out what you want to sample (for example, a
language, genre or subgenre) and taken a representative sample of
it. You must therefore know the size of the sample. That is your
<i>n</i>, which you will use to calculate the statistical
significance of the difference between the two values, probably
with Student's <i>t</i>-test.<br>
<br>
It is the <i>t-</i><tt>test that will tell you whether </tt>your
sample is too small to rule out the possibility that you just
sampled the wrong things. You also need your hypothesis: if you
were expecting the values of System 02 to be higher, you want a
one-tailed <i>t</i><tt>-test; otherwise you want a two-tailed
test.</tt><br>
<br>
If you don't actually have a representative sample or a
hypothesis, then you're just playing an elaborate game of pretend
with your reviewers, where you pretend to find significance and
they pretend to be impressed. In that case, I'm not sure exactly
what you have to pretend to do; maybe someone else can fill us in.<br>
<br>
<a class="moz-txt-link-freetext" href="http://grieve-smith.com/blog/2014/01/you-cant-get-significance-without-a-representative-sample/">http://grieve-smith.com/blog/2014/01/you-cant-get-significance-without-a-representative-sample/</a><br>
<br>
On 11/10/2014 6:28 AM, Jack Alan wrote:<br>
</div>
<blockquote
cite="mid:CAPgQO-L8Vi6NDrbAUfeLdEavtrbw3jJ+UWjj-oz2uZonLupyEQ@mail.gmail.com"
type="cite">
<div dir="ltr">Hi folks,
<div><br>
</div>
<div>A bit struggling of calculating the statistical significant
between the output of two systems. Suppose Ive got the
following two results from two independent systems (performing
sequence labelling task):</div>
<div><br>
</div>
<div>System 01: </div>
<div>precision: 81.57%; recall: 57.12%; FB1: 67.19%<br>
</div>
<div><br>
</div>
<div>
<div>System 02: </div>
<div>precision: 84.07%; recall: 62.47%; FB1: 71.68%</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Could someone pinpoint me to the way of calculating the
statistical significant between them?</div>
<div><br>
</div>
<div>p.s. I've no folds applied (just one go "training and
test")</div>
<div><br>
</div>
<div>J.</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
UNSUBSCRIBE from this page: <a class="moz-txt-link-freetext" href="http://mailman.uib.no/options/corpora">http://mailman.uib.no/options/corpora</a>
Corpora mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Corpora@uib.no">Corpora@uib.no</a>
<a class="moz-txt-link-freetext" href="http://mailman.uib.no/listinfo/corpora">http://mailman.uib.no/listinfo/corpora</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
-Angus B. Grieve-Smith
<a class="moz-txt-link-abbreviated" href="mailto:grvsmth@panix.com">grvsmth@panix.com</a>
</pre>
</body>
</html>