24.4000, FYI: Benchmark for Open Relation Extraction

[linguist at linguistlist.org]

LINGUIST List: Vol-24-4000. Fri Oct 11 2013. ISSN: 1069 - 4875.

Subject: 24.4000, FYI: Benchmark for Open Relation Extraction

Moderator: Damir Cavar, Eastern Michigan U <damir at linguistlist.org>

Reviews: Veronika Drake, U of Wisconsin Madison
Monica Macaulay, U of Wisconsin Madison
Rajiv Rao, U of Wisconsin Madison
Joseph Salmons, U of Wisconsin Madison
Mateja Schuck, U of Wisconsin Madison
Anja Wanner, U of Wisconsin Madison
       <reviews at linguistlist.org>

Homepage: http://linguistlist.org

Do you want to donate to LINGUIST without spending an extra penny? Bookmark
the Amazon link for your country below; then use it whenever you buy from
Amazon!

USA: http://www.amazon.com/?_encoding=UTF8&tag=linguistlist-20
Britain: http://www.amazon.co.uk/?_encoding=UTF8&tag=linguistlist-21
Germany: http://www.amazon.de/?_encoding=UTF8&tag=linguistlistd-21
Japan: http://www.amazon.co.jp/?_encoding=UTF8&tag=linguistlist-22
Canada: http://www.amazon.ca/?_encoding=UTF8&tag=linguistlistc-20
France: http://www.amazon.fr/?_encoding=UTF8&tag=linguistlistf-21

For more information on the LINGUIST Amazon store please visit our
FAQ at http://linguistlist.org/amazon-faq.cfm.

Editor for this issue: Uliana Kazagasheva <uliana at linguistlist.org>
================================================================  


Date: Fri, 11 Oct 2013 22:43:31
From: Filipe Mesquita [mesquita at ualberta.ca]
Subject: Benchmark for Open Relation Extraction

E-mail this message to a friend:
http://linguistlist.org/issues/emailmessage/verification.cfm?iss=24-4000.html&submissionid=21127430&topicid=6&msgnumber=1
 
We wish to announce the public release of the Open Relation Extraction (ORE) benchmark used for the experiments reported in the paper: 

Effectiveness and Efficiency of Open Relation Extraction, by Filipe Mesquita, Jordan Schmidek and Denilson Barbosa, appearing at the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP).

ORE is the task of recognizing relationships between two or more entities in text without requiring any relation-specific training data. ORE has become prevalent over traditional relation extraction methods, especially on the Web, because of the intrinsic difficulty in training individual extractors for every single relation.

To the best of our knowledge, our benchmark is the first of its kind to provide reusable gold standard annotations. Included in the benchmark are over 15,000 annotations (of which 13,000 were done automatically by matching facts in a knowledge base), including binary relations and n-ary relations. We also provide extractions from 8 state-of-the-art ORE methods and evaluation scripts that compute precision and recall of a given set of extractions.

For more information and access to the benchmark itself, please visit the following URL:

https://sites.google.com/a/ualberta.ca/sonex/

--
Filipe Mesquita
Ph.D. Student
Computing Science Department
University of Alberta
http://webdocs.cs.ualberta.ca/~mesquita/ 



Linguistic Field(s): Computational Linguistics
                     Semantics





 






----------------------------------------------------------
LINGUIST List: Vol-24-4000	
----------------------------------------------------------