[Corpora] [Corpora-List] Calculating statistical significant

[Angus Grieve-Smith]

     Good question, Jack!

     You're missing some information here.  As you know, statistical 
significance tells you whether your sample is large enough that the 
effect you observe is probably not due to a bad sample.

http://grieve-smith.com/blog/2014/01/how-big-a-sample-do-you-need/

     So you've figured out what you want to sample (for example, a 
language, genre or subgenre) and taken a representative sample of it.  
You must therefore know the size of the sample.  That is your /n/, which 
you will use to calculate the statistical significance of the difference 
between the two values, probably with Student's /t/-test.

     It is the /t-/test that will tell you whether your sample is too 
small to rule out the possibility that you just sampled the wrong 
things.  You also need your hypothesis: if you were expecting the values 
of System 02 to be higher, you want a one-tailed /t/-test; otherwise you 
want a two-tailed test.

     If you don't actually have a representative sample or a hypothesis, 
then you're just playing an elaborate game of pretend with your 
reviewers, where you pretend to find significance and they pretend to be 
impressed.  In that case, I'm not sure exactly what you have to pretend 
to do; maybe someone else can fill us in.

http://grieve-smith.com/blog/2014/01/you-cant-get-significance-without-a-representative-sample/

On 11/10/2014 6:28 AM, Jack Alan wrote:
> Hi folks,
>
> A bit struggling of calculating the statistical significant between 
> the output of two systems. Suppose Ive got the following two results 
> from two independent systems (performing sequence labelling task):
>
> System 01:
> precision:  81.57%; recall:  57.12%; FB1:  67.19%
>
> System 02:
> precision:  84.07%; recall:  62.47%; FB1:  71.68%
>
>
> Could someone pinpoint me to the way of calculating the statistical 
> significant between them?
>
> p.s. I've no folds applied (just one go "training and test")
>
> J.
>
>
> _______________________________________________
> UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
> Corpora mailing list
> Corpora at uib.no
> http://mailman.uib.no/listinfo/corpora


-- 
				-Angus B. Grieve-Smith
				grvsmth at panix.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20141110/8022def7/attachment.htm>
-------------- next part --------------
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora