[Corpora-List] Seeking for a free comparable corpus

[Diana Santos]

Hi Darren
No, articles from Wikipedia in different languages are NOT a comparable
corpus, for many reasons.

First, most of the time they are a (more or less free) translation of a
master/initial one.
Second, they are about the same (narrow) subject, while a comparable corpus
would be about the same theme but different many  subjects. Examples of
comparable corpora would be: original articles in two languages about
violations of human rights; or about fashion, or about complaints about
health system facilities.

If you are interested in CLIR you could try the CLEF collections which were
precisely created for this.

Second, a parallel corpus is not defined in terms of SENTENCE alignment,
unit is a parameter for parallel. So a Wikipedia collection as the one you
suggest is a parallel corpus where the unit is the wikipedia article, not
the sentence.

Paralell means in a nutshell that you can put the units in direct
corespondence (most of them), while comparable means that the selection
criteria are the same, but you cannot pair the elements of the two coprora.

I hope to have helped.
Best
Diana


2014-06-14 16:15 GMT+02:00 Darren Cook <darren at dcook.org>:

> > I'm working on Cross Language Information Retrieval based on
> > comparable corpora. In order to test my approach, I need a free
> > comparable corpus between English language and an European language.
>
> I was just trying to understand the difference between "parallel corpus"
> and "comparable corpus". Am I correct in thinking that if an article is
> translated (by a professional human translator, or a machine) from one
> language to another, such that there is a sentence-level correspondence,
> then it is a parallel corpus. Whereas a comparable corpus is one where
> the two articles were written about the same subject, but neither is a
> translation of the other, and mostly the same knowledge is covered, but
> a sentence-level mapping would not exist?
>
> If so, Wikipedia sounds like an ideal source.
> E.g.
>   http://en.wikipedia.org/wiki/Paris
>   http://fr.wikipedia.org/wiki/Paris
>
>   http://en.wikipedia.org/wiki/Association_football
>   http://fr.wikipedia.org/wiki/Football
>
> etc.
>
> Darren
>
>
> --
> Darren Cook, Software Researcher/Developer
> My new book: Data Push Apps with HTML5 SSE
> Published by O'Reilly: (ask me for a discount code!)
>   http://shop.oreilly.com/product/0636920030928.do
> Also on Amazon and at all good booksellers!
>
> _______________________________________________
> UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
> Corpora mailing list
> Corpora at uib.no
> http://mailman.uib.no/listinfo/corpora
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listserv.linguistlist.org/pipermail/corpora/attachments/20140614/1851b449/attachment.htm>
-------------- next part --------------
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora