[Corpora-List] Precision and Recall

[Angus Grieve-Smith]

On Sat, 19 Apr 2008, Daniel Zeman wrote:

> the false positives/negatives are absolute numbers. If you evaluate, say, 
> performance of a parser on two different data sets and you get fp=100 and 
> fn=100 for both, you still cannot say that both sets are equally hard for the 
> parser. It may well be that the sets were not the same size and that tp1=100 
> while tp2=1000.

 	Okay, I see that you would want to know how many false negatives 
there are as a proportion - i.e. how many of the positives it found 
correctly - so I see the value in "recall," even if it doesn't make much 
sense as a name.  But it seems to me that the raw number of false 
negatives is also valuable.

 	But false positives are false positives; why does it matter how 
many true positives there were?  Because it's a measure of how muddy the 
water is?  It seems like here, absolute numbers of false positives would 
be more valuable in many situations.  As Google found, it often doesn't 
matter how many false positives you have, as long as the most valuable 
true positives are close to the top of the list.

 	Incidentally, this is not a purely academic line of questioning; I 
worked on an information retrieval project that failed in part because 
precision and recall did not accurately predict customer satisfaction.

 					-Angus B. Grieve-Smith
 					grvsmth at panix.com

_______________________________________________
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora