19.586, FYI: 360,000,000 word BYU Corpus of American English

Wed Feb 20 16:31:24 UTC 2008

LINGUIST List: Vol-19-586. Wed Feb 20 2008. ISSN: 1068 - 4875.

Subject: 19.586, FYI: 360,000,000 word BYU Corpus of American English

Moderators: Anthony Aristar, Eastern Michigan U <aristar at linguistlist.org>
            Helen Aristar-Dry, Eastern Michigan U <hdry at linguistlist.org>

Reviews: Randall Eggert, U of Utah  
         <reviews at linguistlist.org> 

Homepage: http://linguistlist.org/

The LINGUIST List is funded by Eastern Michigan University, 
and donations from subscribers and publishers.

Editor for this issue: Matthew Lahrman <matt at linguistlist.org>
================================================================  

To post to LINGUIST, use our convenient web form at
http://linguistlist.org/LL/posttolinguist.html.

===========================Directory==============================  

1)
Date: 20-Feb-2008
From: Mark Davies < mark_davies at byu.edu >
Subject: 360,000,000 word BYU Corpus of American English

-------------------------Message 1 ---------------------------------- 
Date: Wed, 20 Feb 2008 11:29:50
From: Mark Davies [mark_davies at byu.edu]
Subject: 360,000,000 word BYU Corpus of American English
E-mail this message to a friend:
http://linguistlist.org/issues/emailmessage/verification.cfm?iss=19-586.html&submissionid=170121&topicid=6&msgnumber=1  

We are pleased to announce the release of the 360+ million word ''BYU
Corpus of American English'' (1990-2007), which is freely available online
(http://www.americancorpus.org). New texts will be added at least two times
each year from this point on (20 million new words each year; 4 million
words in each of the five genres), and it will thus serve as a unique
linguistic history of American English since 1990.

CONTENT

The corpus is composed of more than 360 million words in nearly 150,000
texts, including 20 million words each year from 1990-2007. For each year
(and therefore overall, as well), the corpus is evenly divided between the
five genres of spoken, fiction, popular magazines, newspapers, and academic
journals. The texts come from a variety of sources:

Spoken: (76+ million words) Transcripts of unscripted conversation from
nearly 150 different TV and radio programs (examples: All Things Considered
(NPR), Newshour (PBS), Good Morning America (ABC), Today Show (NBC), 60
Minutes (CBS), Hannity and Colmes (Fox), Jerry Springer, etc). 

Fiction: (70 million words) Short stories and plays from literary
magazines, children's magazines, popular magazines, first chapters of first
edition books 1990-present, and movie scripts. 

Popular Magazines: (78+ million words) Nearly 100 different magazines, with
a good mix (overall, and by year) between specific domains (news, health,
home and gardening, women, financial, religion, sports, etc). A few
examples are Time, Men's Health, Good Housekeeping, Cosmopolitan, Christian
Century, Fortune, Sports Illustrated, etc. 

Newspapers: (73+ million words) Ten newspapers from across the US,
including: USA Today, New York Times, Atlanta Journal Constitution, San
Francisco Chronicle, etc. There is also a good mix between different
sections of the newspapers, such as local news, opinion, sports, financial,
etc. 

Academic Journals: (73+ million words) Nearly 100 different peer-reviewed
journals. These were selected to cover the entire range of the Library of
Congress classification system (e.g. a certain percentage from B
(philosophy, psychology, religion), D (world history), K (education), T
(technology), etc.), both overall and by number of words per year 

QUERIES

-- The interface is the same as the interface for the 100 million word
British National Corpus and 100 million word TIME Magazine corpus (see
http://corpus.byu.edu/)

-- Queries by word, phrase, alternates, substring, part of speech, lemma,
synonyms (see below), and customized lists (see below) 

-- The corpus is tagged by CLAWS, the same tagger that was used for the BNC
and the TIME corpus

-- Chart listings (totals for all matching forms in each genre or year,
1990-present, as well as for sub-genres) and table listings (frequency for
each matching form in each genre or year) 

-- Full collocates searching (up to ten words left and right of node word) 

-- Comparisons between genres or time periods (e.g. collocates of 'chair'
in fiction or academic, nouns with 'break the [N]' in newspapers or
academic, adjectives that occur primarily in sports magazines, or verbs
that are more common 2004-2007 than previously)

-- One-step comparisons of collocates of related words, to study semantic
or cultural differences between words (e.g. comparison of collocates of
'small' and 'little', or 'men' and 'women', or 'rob' vs 'steal')

-- Include semantic information from a 60,000 entry thesaurus directly as
part of the query syntax (e.g. frequency and distribution of synonyms of
'beautiful', synonyms of 'strong' occurring in fiction but not academic,
synonyms of 'clean' + noun ('clean the floor', 'washed the dishes')

-- Create your own 'customized' word lists, and then re-use these as part
of subsequent queries (e.g. lists related to a particular semantic category
(clothes, foods, emotions), or a user-defined part of speech)

NOTE

Due to copyright and licensing issues, the corpus is not available in
full-text form. Rather, as with our interface to the BNC and TIME, all
access will be via the web interface, which allows full frequency and
distributional charts, and limited KWIC displays (up to 100 words per entry) 

============================================
Mark Davies
Professor of (Corpus) Linguistics
Brigham Young University
(phone) 801-422-9168 / (fax) 801-422-0906
Web: davies-linguistics.byu.edu

** Corpus design and use // Linguistic databases **
** Historical linguistics // Language variation **
** English, Spanish, and Portuguese **
============================================ 

Linguistic Field(s): Computational Linguistics
                     Lexicography
                     Text/Corpus Linguistics

-----------------------------------------------------------
LINGUIST List: Vol-19-586	