[Corpora-List] WSD / # WordNet senses / Mechanical Turk

[Angus Grieve-Smith]

On 7/16/2013 2:43 AM, Adam Kilgarriff wrote:
> For either automatic WSD, or even for the gold standard, I agree 
> entirely with John:
>
>     Miss Elliott, my high-school English teacher, wouldn't give
>     anyone a gold star [for work like that]
>
>

     Well, now hang on a minute.

     First of all, how unambiguous were these, originally?  Real 
language is full of ambiguous uses.  I'm surprised that the experts can 
even agree 95% of the time, and I'd guess that part of their expert 
training was aimed at exaggerating that agreement to satisfy brittle 
models that weren't built to handle persistent ambiguity.

     Second, if the human experts didn't even agree, why would Miss 
Elliott expect her students to do any better?  What kind of a sadist was 
she?

-- 
				-Angus B. Grieve-Smith
				grvsmth at panix.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20130716/1470e149/attachment.htm>
-------------- next part --------------
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora