[Corpora-List] Brown Corpus

[Lou Burnard]

Well, every generation has every right to reinvent the work of their
predecessors, but such a  "rationalization" seems to me to play somewhat
fast and loose with the design chosen by the original compilers of the
Brown Corpus... it was intended to comprise 500 equally-sized samples,
selected from 15 pre-defined categories.  By lumping together all the
samples from the same category you will wind up with differently sized
samples -- category J ("learned") has 80 texts, c. 160,000 words; while
category R ("humor") has only 9, i.e. 18,000 words.  That may not
matter, of course, for many applications, but it seems a shame to lose
that feature of the original design. Or will you preserve the original
text boundaries with some additional markup?
If so, you might like to consider that many of the original 2000 word
samples have some internal structure too: even within 2000 words, there
are samples taken from originally discontinuous sections of the
original.  Which most versions seem to disregard.

antiquariously,

Lou


Steven Bird wrote:

>Note that this version of the Brown Corpus contains 500 files, each
>consisting of around 200 lines of text on average.  Perhaps these were
>as big as they could handle back in 1961.  I think it would make matters
>simpler if the file structure was rationalized now, so that, e.g.:
>
>Brown Corpus file names
>Existing     -> Proposed
>ca01 .. ca44 -> a
>cb01 .. cb26 -> b
>etc
>
>(NB this is how things are being restructured in NLTK-Lite, a new,
>steamlined version of NLTK that will be released later this month.)
>
>-Steven Bird
>
>
>On Tue, 2005-06-14 at 17:27 +0100, Lou Burnard wrote:
>
>
>>By one of those uncanny coincidences, I am planning to include an
>>XMLified version of the Brown corpus on the next edition of the BNC Baby
>>corpus sampler. The version I have is derived from the GPLd version
>>distributed as part of the LTK tool set (http://nltk.sourceforge.net)
>>and includes POS tagging; there is also a version which has been
>>enhanced to include Wordnet semantic tagging but I am not clear as to
>>the rights in that.
>>
>>Lou Burnard
>>
>>
>>Xiao, Zhonghua wrote:
>>
>>
>>>The plain text version of Brown is available here:
>>>http://dingo.sbs.arizona.edu/~hammond/ling696f-sp03/browncorpus.txt
>>>
>>>Richard
>>>________________________________
>>>
>>>From: owner-corpora at lists.uib.no on behalf of Jörg Schuster
>>>Sent: Tue 14/06/2005 14:39
>>>To: CORPORA at hd.uib.no
>>>Subject: [Corpora-List] Brown Corpus
>>>
>>>
>>>
>>>Hello,
>>>
>>>where can the Brown Corpus be downloaded or purchased?
>>>
>>>Jörg Schuster
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>
>
>
>
>