<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#663300">
<font face="Cambria">Hi Mark<br>
<br>
I am sure you are aware of the work conducted within the CLEF
project (</font>Cross-Language Evaluation Forum)<br>
<font face="Cambria">A large part of the evaluation is about
Authorship and addresses this from various angles (Plagiarism,
"Vandalism", Authorship identification/attribution, etc.)<br>
see details of the PAN lab at:
<a class="moz-txt-link-freetext" href="http://clef2011.org/index.php?page=pages/labs_program.html">http://clef2011.org/index.php?page=pages/labs_program.html</a><br>
<br>
</font>Hope this helps<br>
<br>
Khalid<br>
<br>
<br>
Justin Washtell wrote, On 17/04/2012 23:17:
<blockquote
cite="mid:B963733A3B7F9845815FF33B96FFDCBA01B2E693F545@HERMES8.ds.leeds.ac.uk"
type="cite">
<pre wrap="">Hi Mark,
Statistics such as Log-Likelihood (see <a class="moz-txt-link-freetext" href="http://ucrel.lancs.ac.uk/llwizard.html">http://ucrel.lancs.ac.uk/llwizard.html</a>), can give an indication of how significant are differences in observed freqeuencies of events.
These sorts of statistics assume a null-hypothesis in which everyhing is entirely random or unrelated, outside of which things are considered to be "significant". You need to be careful with this. Often in reality - as in your case I think - what you are looking for is actually more subtle.
For example, I would suggest that you will at least want to look at similar n-gram statistics derived from all other pairwise combinations of chapters in your particular corpus, to establish whether what is observed between A and B is somehow "special" in your case.
Also, I imagine the observed frequencies of those lower order n-grams which constitute your longer n-grams will have a bearing on how remarkable the figures are before you even start looking at the relative differences. For getting a handle on that, the language modelling literature may be useful.
Sorry I can not be more specific. I'm not a statistician :-)
Justin Washtell
University of Leeds
________________________________________
From: <a class="moz-txt-link-abbreviated" href="mailto:corpora-bounces@uib.no">corpora-bounces@uib.no</a> [<a class="moz-txt-link-abbreviated" href="mailto:corpora-bounces@uib.no">corpora-bounces@uib.no</a>] On Behalf Of Yorick Wilks [<a class="moz-txt-link-abbreviated" href="mailto:Y.Wilks@dcs.shef.ac.uk">Y.Wilks@dcs.shef.ac.uk</a>]
Sent: 17 April 2012 21:03
To: Mark Davies
Cc: <a class="moz-txt-link-abbreviated" href="mailto:corpora@uib.no">corpora@uib.no</a>
Subject: Re: [Corpora-List] Comparing n-grams / authorship
The questioner might want to look at the METER project: <a class="moz-txt-link-freetext" href="http://aclantho3.herokuapp.com/catalog/P02-1020">http://aclantho3.herokuapp.com/catalog/P02-1020</a>
This was an attempt to determine if one text had been rewritten from another based on ngrams---in a journalism and press service context (rather than plagiarism). it turned out that such texts could have very long ngrams in common without having been rewritten from ecah other.
Yorick Wilks
On 17 Apr 2012, at 15:47, Mark Davies wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I am sending the following question on behalf of a colleague at BYU. Thanks in advance for any suggestions you have; I'll forward them to the researcher who is working on this problem.
Mark Davies, BYU
-------------------------------------------
I am working with a 250,000 word text. Within this text there are two chapters, A and B (1,200 and 2,400 words respectively). The authorship of these two chapters is unknown, but we have reason to believe to that the author(s) of A and B have a relationship that is different from the majority of the rest of the book. There are two 4-grams, three 6-grams, one 7-gram, one 8-gram, and one 9-gram shared in common in chapters A and B that appear nowhere else in the book. Intuitively it seems like there is a unique relationship between chapters A and B.
The question is:
Is there a statistical method of measuring whether the types of n-grams above establish a reasonable probability that the two texts are linked.
_______________________________________________
UNSUBSCRIBE from this page: <a class="moz-txt-link-freetext" href="http://mailman.uib.no/options/corpora">http://mailman.uib.no/options/corpora</a>
Corpora mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Corpora@uib.no">Corpora@uib.no</a>
<a class="moz-txt-link-freetext" href="http://mailman.uib.no/listinfo/corpora">http://mailman.uib.no/listinfo/corpora</a>
</pre>
</blockquote>
<pre wrap="">
_______________________________________________
UNSUBSCRIBE from this page: <a class="moz-txt-link-freetext" href="http://mailman.uib.no/options/corpora">http://mailman.uib.no/options/corpora</a>
Corpora mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Corpora@uib.no">Corpora@uib.no</a>
<a class="moz-txt-link-freetext" href="http://mailman.uib.no/listinfo/corpora">http://mailman.uib.no/listinfo/corpora</a>
_______________________________________________
UNSUBSCRIBE from this page: <a class="moz-txt-link-freetext" href="http://mailman.uib.no/options/corpora">http://mailman.uib.no/options/corpora</a>
Corpora mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Corpora@uib.no">Corpora@uib.no</a>
<a class="moz-txt-link-freetext" href="http://mailman.uib.no/listinfo/corpora">http://mailman.uib.no/listinfo/corpora</a>
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<b> Khalid Choukri </b>
<br>
ELRA General secretary & ELDA CEO
<br>
email: <a class="moz-txt-link-abbreviated" href="mailto:choukri@elda.org">choukri@elda.org</a>; <br>
Web: <a class="moz-txt-link-abbreviated" href="http://www.elra.info">www.elra.info</a> <a class="moz-txt-link-abbreviated" href="http://www.elda.org">www.elda.org</a>
<br>
Tel. +33 1 43 13 33 33 - Fax. +33 1 43 13 33 30
<br>
<br>
<b> ***************************************************<br>
** Info on LREC 2012 : <a class="moz-txt-link-abbreviated" href="http://www.lrec-conf.org">www.lrec-conf.org</a> <br>
***************************************************<br>
</b></div>
</body>
</html>