[Corpora-List] Workshop: (Learner) Corpora and their application in languagetesting and assessment

[Sandra Götz]

Pre-conference workshop to be held at ICAME 34
"English corpus linguistics on the move: Applications and implications"
Santiago de Compostela, Spain
Wednesday, May 22nd 2013


(Learner) Corpora and their application in language testing and assessment

Corpora and corpus linguistic tools and methods are frequently used in  
the study of second language (L2) learning, most notably in Learner  
Corpus Research (LCR). LCR has contributed significantly to the  
description of interlanguages and many of its findings have resulted  
in useful applications for foreign language teaching and learning.  
Learner- and native-speaker corpora have also received increasing  
attention in the area of language testing and assessment (LTA; Barker  
2010; Taylor & Barker 2008). Practical applications of corpora in LTA  
can range from corpus-informed to corpus-based and corpus-driven  
approaches, depending on how corpus data are actually put into  
practice, the aims and outcomes for LTA, and the degree of involvement  
of the researcher in the process of data retrieval, analysis and  
interpretation (Barker 2010; Callies, Zaytseva & Diez-Bedmar to appear).

More recently, researchers have also turned to corpora to inform,  
validate, and develop the way proficiency is operationalized in the  
Common European Framework of Reference for Languages (CEFR; Council of  
Europe 2001, 2009). While the CEFR has been highly influential in  
language testing and assessment, the way it defines proficiency levels  
using "can-do-statements" has been criticized, because they are often  
too impressionistic. For example, a learner at the C2 level is  
expected to maintain "consistent grammatical control of complex  
language", whereas at C1 he/she should "consistently maintain a high  
degree of grammatical accuracy" (Council of Europe 2001, 2009). Such  
global, vague and underspecified descriptions have limited practical  
value to distinguish between proficiency levels and also fail to give  
in-depth linguistic details regarding individual languages or  
learners' skills in specific registers. These shortcomings have led to  
an increasing awareness among researchers of the need to identify more  
specific linguistic descriptors or 'criterial features' which can be  
quantified by learner data. The aim of such corpus-based approaches is  
to add "grammatical and lexical details of English to CEFR’s  
functional characterisation of the different levels" (Hawkins &  
Filipovic 2012: 5).

While (learner) corpora have the potential to increase transparency,  
consistency and comparability in the assessment of L2 proficiency,  
several problems and challenges may also be encountered. One major  
difficulty is that "proficiency level" has often been a fuzzy variable  
in learner corpus compilation and analysis (Carlsen 2012), because,  
due to practical constraints, proficiency has mostly been  
operationalized and assessed globally by means of external criteria,  
typically learner-centred methods such as learners' institutional  
status. However, recent studies show that global proficiency measures  
based on external criteria alone are not reliable indicators of  
proficiency for corpus compilation (Mukherjee 2009; Callies to appear  
2013), and "hidden" differences in proficiency (e.g. Pendar & Chapelle  
2008) often go undetected or tend to be disregarded in learner corpus  
analysis (e.g. Götz 2013). Thus, the field still seems to be in need  
of a corpus-based description of language proficiency to account for  
inter-learner variability and seek homogeneity in learner corpus  
compilation and L2 assessment. Another issue that has been intensively  
debated is the appropriate basis of comparison for learner corpus  
data, i.e. against what yardstick learner performance should be  
compared and evaluated.

The aim of this workshop is to discuss the benefits in terms of  
current practices and developments, but also the challenges and  
possible obstacles of using both native-speaker reference corpora and  
learner corpora for testing and assessing L2 proficiency. We thus  
invite submissions that provide case studies exemplifying how corpora  
can be used for the assessment of L2 proficiency in both speaking and  
writing. In particular, submissions should address one of the  
following topics:

- corpus compilation (types of corpus data and their usefulness for  
testing purposes; proficiency as a fuzzy variable in learner corpus  
compilation and analysis; homogeneity vs. variability in corpus  
composition)

- corpus comparability (e.g. as to register/genre or task setting and  
conditions, i.e. testing vs. non-testing contexts, prompt, timing,  
access to reference works)

- the operationalization of (types of) proficiency in corpus  
approaches to testing and assessment

- the use of corpora in data-driven approaches to the assessment of  
proficiency (e.g. using corpus data to validate or complement human  
rating as in studies based on error-tagged learner corpora, or using  
corpus data (partially) independently of human rating).


Abstracts should be 400 to 500 words long (excluding references). They  
should be submitted by e-mail to callies at uni-bremen.de and  
Sandra.Goetz at anglistik.uni-giessen.de by 1st February 2013.  
Notification of acceptance will be sent out in late February 2013.

A PDF-version of this CFP can be downloaded here:

http://www.usc.es/export/sites/default/en/congresos/icame34/descargas/WS-CalliesGotz.pdf

References

Barker, F. (2010), How can corpora be used in language testing? In A.  
O'Keeffe & M. McCarthy (eds.) The Routledge Handbook of Corpus  
Linguistics (pp. 633-645). New York: Routledge.

Callies, M. (to appear 2013), Advancing the research agenda of  
Interlanguage Pragmatics: The role of learner corpora. Yearbook of  
Corpus Linguistics and Pragmatics.

Callies, M., Zaytseva, E. & Diez-Bedmar, M.B. (to appear), Using  
learner corpora for testing and assessing L2 proficiency. In P.  
Leclercq, H. Hilton & A. Edmonds (eds.) Proficiency Assessment Issues  
in SLA Research: Measures and Practices. Clevedon: Multilingual Matters.

Carlsen, C. (2012), Proficiency level – a fuzzy variable in computer  
learner corpora. Applied Linguistics 33(2), 161-183.

Council of Europe (2001), Common European Framework of Reference for  
Languages: Learning, teaching, assessment. Cambridge: Cambridge  
University Press.

Council of Europe (2009), Relating Language Examinations to the Common  
European Framework of Reference for Languages: Learning, Teaching,  
Assessment (CEFR): A Manual. Strasbourg: Language Policy Division.

Götz, S. (2013), Fluency in Native and Nonnative English Speech.  
Amsterdam: Benjamins.

Hawkins, J. & Filipović, L. (2012), Criterial Features in L2  
English. Cambridge: Cambridge University Press.

Mukherjee, J. (2009), The grammar of conversation in advanced spoken  
learner English: Learner corpus data and language-pedagogical  
implications. In K. Aijmer (ed.), Corpora and Language Teaching (pp.  
203-230). Amsterdam: Benjamins.

Pendar, N. & Chapelle, C.A. (2008), Investigating the promise of  
learner corpora: Methodological issues. CALICO Journal 25, 189-206.

Taylor, L. & Barker, F. (2008), Using corpora for language assessment.  
In E. Shohamy & N.H. Hornberger (eds.), Encyclopedia of Language and  
Education, 2nd edition, volume 7: Language testing and assessment (pp.  
241-254). New York: Springer.



_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora