<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=us-ascii">
<META content="MSHTML 6.00.3790.2541" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=844500518-15032006><FONT face=Arial
color=#0000ff size=2>Discourse structure theory may be an appropriate tool for
this job. However, Rhetorical Structure Theory is unlikely to be the
d<SPAN class=844500518-15032006><FONT face=Arial color=#0000ff size=2>iscourse
structure theory that helps. It's rather ad hoc (and I'm being charitable
here). I'd look at work by Livia Polanyi and work on Discourse
Representation Theory. Someone actually familiar with the field could
probably make stronger recommendations.</FONT></SPAN></FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=844500518-15032006><FONT face=Arial
color=#0000ff size=2><SPAN
class=844500518-15032006></SPAN></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=844500518-15032006><FONT face=Arial
color=#0000ff size=2><SPAN class=844500518-15032006>Max
Copperman</SPAN></FONT></SPAN></DIV><BR>
<DIV class=OutlookMessageHeader lang=en-us dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>From:</B> owner-corpora@lists.uib.no
[mailto:owner-corpora@lists.uib.no] <B>On Behalf Of </B>Alexander
Schutz<BR><B>Sent:</B> Wednesday, March 15, 2006 9:30 AM<BR><B>To:</B>
D.G.Damle<BR><B>Cc:</B> CORPORA<BR><B>Subject:</B> Re: [Corpora-List] Author+'s
plans for books<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">
<DIV style="DIRECTION: ltr">
<P><FONT face=Arial size=2>I am trying to learn ontologies from text.
Evaluation is a problem, since if you ask people to read the text and then to
evaluate the automatically generated ontology; every reader's concept
structure may be different. The variation amongst readers may be too
great! </FONT></P></DIV></BLOCKQUOTE>
<DIV>In my opinion, it will be extremely helpful to restrict the amount of
concepts (or the choice of concepts in general). It is not so obvious what you
are trying to achieve: <BR>Evaluating the learned concepts of a system against a
gold standard? Then, on which kind of corpus did you conduct your experiments? I
assume it is a domain specific corpus (of textbooks). In that case it would be
quite easy to agree on a subset of certain concepts for that domain, and
restrict the domain experts (readers) to refer only to elements of this subset
while evaluating your system.<BR></DIV>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">
<DIV style="DIRECTION: ltr">
<P><FONT face=Arial size=2>It is also difficult to have such an ontology
marked by domain experts. What the domain experts know about the domain
may not be reflected in the text and so Rrecall is particularly
difficult. Also, evaluators may not be willing to read large
texts.</FONT></P></DIV></BLOCKQUOTE>
<DIV>Evaluation in ontology learning is a pain in the neck, and your problem
with precision will by far outweigh your recall problem. Just imagine that your
goal is to *learn* ontology concepts (or relations). What if your system is
learning something new (i.e. which is not contained in the gold standard, or in
your subset of concepts agreed upon?). It will then contribute to your precision
error.<BR>On the other hand, if you decide to compose your gold standard of all
the possible concepts in the whole world (just to make sure your system will not
run into precision problems described above), there will be loads of concepts
that you miss, because they are not contained in the text (which accounts for
the recall problem you described). Yes, evaluation of ontology learning, it is a
dilemma.<BR><BR>The fact that evaluators may not be willing to read large texts
is in my opinion not a problem of ontology learning and there is a lot you can
do to assure the loyalty of your evaluators (hint hint)<BR></DIV>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">
<DIV style="DIRECTION: ltr">
<P><FONT face=Arial size=2>Does the ontology defined by the author(s) of a
large text constitute a more objective yardstick? Do authors have a list
of concepts and possibly some notion of structure about the text they set out
to create? (I am thinking particularly of textbooks). Do any authors
commit something like a concept structure to paper or a computer
documentbefore they write the text? Alternatively, is it likely that an
author could retrospectively construct such a plan, notwithstanding the
issues of memory lapses etc.</FONT></P></DIV></BLOCKQUOTE>
<DIV>To be honest I have not written any textbook but I would like to think that
before I write a larger chunk of text (say a paper), I have a certain structure
(and the containing concepts so to speak) in mind before I actually start
writing.<BR></DIV>
<BLOCKQUOTE class=gmail_quote
style="PADDING-LEFT: 1ex; MARGIN: 0pt 0pt 0pt 0.8ex; BORDER-LEFT: rgb(204,204,204) 1px solid">
<DIV style="DIRECTION: ltr">
<P><FONT face=Arial size=2>Do any authors have such plans and the texts they
wrote using those plans in an electronic form which they would be happy to
make available for research? What do list members who write textbooks,
do?</FONT></P></DIV></BLOCKQUOTE>
<DIV>If you speak of text planning, then maybe discourse and text theory is the
right thing for you, such as Rhetorical Structure Theory<BR><PRE>@Article{thompson-mann87,<BR> Author="Thompson, Sandra A. and Mann, William C."<BR> Title="Rhetorical Structure Theory: A framework for the analysis of texts",<BR> Journal="IPrA Papers in Pragmatics",
<BR> Volume=1,<BR> Number=1,<BR> Pages="79-105",<BR> Abstract="One of the foundation papers of RST."<BR> Year=1987}<BR></PRE></DIV></DIV><BR>--
<BR>Alexander Schutz<BR>Student of Computational Linguistics<BR>University of
Saarland, Germany </BODY></HTML>