[EDLING:2070] EducationGuardian.co.uk: Flexibility is proof of a good 'Framework'

[fmhult at DOLPHIN.UPENN.EDU]

Francis Hult spotted this on the EducationGuardian.co.uk site and thought you should see it.

-------
Note from Francis Hult:

"The Common European Framework of Reference (CEFR) provides guidance to agencies and governments interested in language teaching and testing. It can also provide help to language teachers who prepare tests and exams regularly, whether as a formal check assessment for progress, as an informal quiz, or even as part of a lesson plan." 

-------

To see this story with its related links on the EducationGuardian.co.uk site, go to http://education.guardian.co.uk

Flexibility is proof of a good 'Framework'
How to bend the rules of test writing with Europe's guide to language ability
Fred Davidson and Glenn Fulcher 
Friday November 17 2006
The Guardian


The Common European Framework of Reference (CEFR) provides guidance to agencies and governments interested in language teaching and testing. It can also provide help to language teachers who prepare tests and exams regularly, whether as a formal check assessment for progress, as an informal quiz, or even as part of a lesson plan.  Teachers who want to link classroom activities to the CEFR - and as it becomes ubiquitous in syllabuses and curricula, more will - can consult the Framework for guidance. For example, the following excerpt from section 7.2.2 provides general guidance on the design of tasks for reading and listening comprehension.  Comprehension tasks may be designed so that the same input may be available to all learners but different outcomes may be envisaged quantitatively (amount of information required) or qualitatively (standard of performance expected).  Alternatively, the input text may contain different amounts of information or degrees of cognitive and/or o
 rganisational complexity, or different amounts of support (visuals, key words, prompts, charts, diagrams, etc) may be made available to help learners. Input may be chosen for its relevance to the learner (motivation) or for reasons extrinsic to the learner.  A text may be listened to or read as often as necessary or limits may be imposed. The type of response required can be quite simple (raise your hand) or demanding (create a new text). In the case of interaction and production tasks, performance conditions can be manipulated in order to make a task more or less demanding . . .  Suppose a teacher is working on listening comprehension. Having consulted this paragraph and designed a listening test, naturally she asks: "How do I know if I've got it right?"  Let's focus on one famously thorny problem in language test design: how many times should a student be permitted to hear the passage? Our teacher notes that the Framework provides great latitude on several matters. After s
 ome deliberation, our teacher writes the following in her background notes:  The CEFR states "A text may be listened to or read as often as necessary or limits may be imposed." (section 7.2.2). In this task, we permit the student to hear the text only twice before answering the comprehension questions.  Our teacher has actually written a part of a test specification. A specification (or "spec") is a blueprint from which actual test tasks can be created. There are many elements in any spec, but all types share two key features: guiding language that describes the task and samples of the task itself. Our teacher has consulted the CEFR, found that it permits variability about the number of times a passage can be heard, made a decision about how many times to hear the passage, and then stated that in writing.  The spec would also (we assume) include sample passages and sample questions, along with additional guiding language about how to write the text, how to form the questions
 , and how to score the results.  The CEFR is not a bank of specifications. It is a high-level set of claims about language ability from which tests and teaching activities can be generated. Between the CEFR and actual implementation should sit specs in which its user sets forth crucial decisions as permitted by the CEFR, as our teacher has done here.  How does our teacher know if her spec is right? As this comprehesion test is used, she may discover that two hearings are too many - or not enough. More likely, our teacher will discover that the number of hearings of any language text depend upon the actual comprehension questions posed about that text and upon the intrinsic characteristics of the text itself. An easy question about a difficult passage might work well on one hearing, but a difficult question on the very same text may require several. This kind of balance is hard to get right the first time, and matching question types to text types is equally challenging.  The
  secret is to do the best one can and remain willing to open up the test for later criticism and change, as data rolls in, as colleagues criticise the exam, and as the teacher's understanding of the CEFR grows and evolves. The wide latitude provided by the Framework is a good thing, so long as some eventual decision about implementation is made - a decision that falls to a spec.  The CEFR has drawn some sharp criticism but its critics have on occasions overlooked one real problem: locking in a CEFR-based test and remaining unwilling to change.  Users of the CEFR should build generative test specs based on it - and this is already happening - but they should be willing to alter those specs as needs merit. CEFR-based specs and the tests they generate should be seen as organic, evolving documents. Gradually, we will feel more and more as if we are getting it right, and in addition, our feedback to the CEFR's authors will become more convincing. Evolving classroom use is superb 
 evidence to alter any standards-based educational system, this Framework included.  · Fred Davidson is associate professor in the division of English as a foreign language at the University of Illinois. Glenn Fulcher is senior lecturer in education (Tesol) in the School of Education, University of Leicester. Click here for more information about the CEFR.

Copyright Guardian News and Media Limited