<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Selon chris brew le 29/03/2010 05:37:<br>
<span style="white-space: pre;">> I also agree with the implicit<br>
> suggestion that keeping markup and text in the same file is not<br>
> always the best idea.<br>
</span><br>
In our projects, one factor to organize the corpus architecture<br>
is to try to separate the parts that change the most often from the
parts<br>
that don't change much (for example several tags - from different<br>
taggers and tag sets - from the surface of texts in NLP projects).<br>
For this, we use various XML standoff annotations techniques.<br>
We also use the one word by line technique for some part of<br>
our workflows (aka IMS CWB source format).<br>
<span style="white-space: pre;"><br>
> it is crucial to document the format as well as you are able,<br>
> and make clear statements about what the annotations are supposed
to<br>
> mean.</span><br>
<br>
We use the guidelines of, and participate to, the Text Encoding
Initiative<br>
(TEI) community : <a class="moz-txt-link-freetext" href="http://www.tei-c.org">http://www.tei-c.org</a>, which documents corpora sources<br>
for that exact purpose since 1994.<br>
If you feel NLP data is not very well represented in that standard, you<br>
are welcome to propose new encodings and discuss their adoption in the<br>
annual update of the guidelines.<br>
For example, we are in a process of proposing new encodings to<br>
document all the history of the various command line tools that were<br>
called during the preparation of a corpus (tokenizers and their<br>
parameters, taggers, etc.). We would like our tools to be able to read<br>
that history for their own processing needs.<br>
Documenting is a must, but sharing that documentation between persons<br>
and softwares is a must also.<br>
<br>
--Serge Heiden<br>
<br>
-- <br>
Dr. Serge Heiden, <a class="moz-txt-link-abbreviated" href="mailto:slh@ens-lyon.fr">slh@ens-lyon.fr</a>, <a class="moz-txt-link-freetext" href="http://textometrie.ens-lsh.fr">http://textometrie.ens-lsh.fr</a><br>
ENS de Lyon/CNRS - ICAR UMR5191, Institut de Linguistique Française<br>
15, parvis René Descartes 69342 Lyon BP7000 Cedex, tél. +33(0)622003883<br>
<br>
</body>
</html>