[Corpora-List] Evaluating abstractive summaries

[John F Sowa]

On 4/28/2013 2:52 AM, Aqil Azmi wrote:
> What is the best tool to automatically evaluate abstractive summaries?

Cautionary note:  I have seen evaluations of summaries that were
done by (a) professional linguists and (b) specialists in the field.

It often happened that the linguists would judge two summaries
as equally good, but the specialists in the subject matter would
have very strong preferences for one in comparison with the other.

For example, the linguists might judge a clear, readable summary
as good, even though it omitted some critical information.  But
the specialists in the subject were more interested in the content.

And sometimes the critical content word(s) might only occur once
in the entire text -- but the specialist would know that they are
key to understanding the main point of the text.

And by the way, abstracts of research papers written by the authors
themselves are usually terrible as summaries of the content.

John Sowa



_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora