agreement in hierarchical structures
Anna Kazantseva
ankazant at SITE.UOTTAWA.CA
Mon Dec 7 19:27:21 UTC 2009
Hi everybody,
It seems that I need to ask for help again. Does anyone know of any
agreement metrics that would be suitable for hierarchical structures
(trees, in my case)? I am particularly interested in metrics that would
compare agreement at different levels.
I am working with a corpus of discourse trees annotated by 2 people and
I want to check the hypothesis that they agree quite well at the leaves
but the agreement worsens as the spans become larger. I consider both
segmentation and labeling of the arcs. I do not seem to find a metric
that would not over-penalize disagreements that are due to disagreement
at lower levels.
I would appreciate any pointers or thoughts.
Thank you very much in advance
Anna
--
Anna Kazantseva, Ph.D. Candidate
School of Information Technology and Engineering
University of Ottawa
800 King Edward Ave. Ottawa, Ontario, Canada, K1N 6N5
More information about the Rstlist
mailing list