<html>
<body>
For statistical tests, I suggest you use a paired two tailed t-test
comparing the rankings with Mean Average Precision or with Precision
measured at rank 10. Other researchers use the Wilcoxon test, which is
also a good test.<br><br>
I co-wrote a paper on statistical tests which appeared in this summer's
SIGIR, you can get it from here<br><br>
<x-tab> </x-tab>
<a href="http://dis.shef.ac.uk/mark/cv/publications/papers/my_papers/SIGIR2005.pdf" eudora="autourl">
http://dis.shef.ac.uk/mark/cv/publications/papers/my_papers/SIGIR2005.pdf</a>
<br><br>
At 15:09 11/10/2005, Chris Jordan wrote:<br>
<blockquote type=cite class=cite cite="">Not to plug my own thesis work
or anything :P<br><br>
I have been exploring the usage of automatic query generation based on
relative entropy to manufacture controlled environments for evaluating
retrieval algorithms. I have paper I am preparing for ECIR with futher
controlled environments that I have been playing with that has resulted
in some interesting findings. I can pass that along to you if it gets
accepted (knock on wood) but for now I can give you ready access to a pdf
of my thesis.<br><br>
<br>
Timad Kahena wrote:<br><br>
<blockquote type=cite class=cite cite="">Hi, <br>
Based on two (or more) results list (in TREC formats), how can we compare
two IR systems using statistical tests? Could someone show me how to do
that? <br>
Examples and tutorial links are welcome…<br><br>
Thank you,<br>
Timad<br><br>
<br>
<x-tab> </x-tab><x-tab>
</x-tab><br>
---------------------------------<br>
Appel audio GRATUIT partout dans le monde avec le nouveau Yahoo!
Messenger<br>
Téléchargez le ici ! <br><br>
<br>
<br>
</blockquote><br><br>
<br>
</blockquote>
<x-sigsep><p></x-sigsep>
<br>
<a href="http://nlp.cs.nyu.edu/hlt-naacl06/" eudora="autourl">
http://nlp.cs.nyu.edu/hlt-naacl06/<br><br>
<br>
</a><font face="Courier, Courier">
____________________________________________________________________<br>
Mark Sanderson, Room
303
Tel: +44 (0) 114 22 22648<br>
Department of Information Studies Fax: +44
(0) 114 27 80300<br>
University of Sheffield, Regent Court,
<a href="mailto:m.sanderson@shef.ac.uk" eudora="autourl">
mailto:m.sanderson@shef.ac.uk<br>
</a>Portobello St, Sheffield, S1 4DP, UK
<a href="http://dis.shef.ac.uk/mark/" eudora="autourl">
http://dis.shef.ac.uk/mark/<br>
</a>
____________________________________________________________________<br>
Good judgement is from experience, experience is from bad judgement<br>
</font></body>
</html>