16.3304, Review: Applied Ling: Cheng, Watanabe & Curtis (2004)

[LINGUIST List]

LINGUIST List: Vol-16-3304. Wed Nov 16 2005. ISSN: 1068 - 4875.

Subject: 16.3304, Review: Applied Ling: Cheng, Watanabe & Curtis (2004)

Moderators: Anthony Aristar, Wayne State U <aristar at linguistlist.org>
            Helen Aristar-Dry, Eastern Michigan U <hdry at linguistlist.org>
 
Reviews (reviews at linguistlist.org) 
        Sheila Dooley, U of Arizona  
        Terry Langendoen, U of Arizona  

Homepage: http://linguistlist.org/

The LINGUIST List is funded by Eastern Michigan University, Wayne
State University, and donations from subscribers and publishers.

Editor for this issue: Lindsay Butler <lindsay at linguistlist.org>
================================================================  

What follows is a review or discussion note contributed to our 
Book Discussion Forum. We expect discussions to be informal and 
interactive; and the author of the book discussed is cordially 
invited to join in. If you are interested in leading a book 
discussion, look for books announced on LINGUIST as "available 
for review." Then contact Sheila Dooley at dooley at linguistlist.org. 

===========================Directory==============================  

1)
Date: 15-Nov-2005
From: Esmat Babaii < ebabaii at gmail.com >
Subject: Washback in Language Testing 

	
-------------------------Message 1 ---------------------------------- 
Date: Wed, 16 Nov 2005 16:42:19
From: Esmat Babaii < ebabaii at gmail.com >
Subject: Washback in Language Testing 
 

EDITORS: Cheng, Liying; Watanabe, Yoshinori; Curtis, Andy
TITLE: Washback in Language Testing
SUBTITLE: Research Contexts and Methods
PUBLISHER: Lawrence Erlbaum Associates
YEAR:  2004
Announced at http://linguistlist.org/issues/16/16-2112.html 

Esmat Babaii, Department of Foreign Languages, University for 
Teacher Education, Tehran, Iran

A decade after the scholarly serious appraisal of the notion of 
washback pioneered by Alderson and Wall (1993), it is timely to have 
a new publication which looks carefully at washback from both 
theoretical and empirical perspectives. Washback in Language 
Testing is a well-balanced and informative collection of various articles 
representing different methodological frameworks and offering 
different positions and research findings about washback in language 
testing. 

The book consists of two parts. Part One, which consists of three 
chapters, introduces the concept of and theoretical arguments about 
washback in language testing and presents the most common 
methodological frameworks used to investigate this complex 
phenomenon. Part Two includes Chapters Four to Eleven. It reports 
the results of several empirical studies on washback. 

In the 'foreword' to this volume, Charles Alderson, while providing a 
useful brief background and explaining how he became involved in 
investigating washback, questions what he calls a "Messickian view" 
(p. xi, see also Messick, 1996) of engineering positive washback by 
test design. He then calls for a multi-dimensional treatment of 
washback phenomenon which considers washback not as a direct 
effect of test in itself, but as a result of the interaction of numerous 
factors existing in the educational system.

In Chapter One: 'Washback or backwash: A review of the impact of 
testing on teaching and learning', Cheng and Curtis offer a collection 
of different outlooks on washback and suggest that instead of being 
very much concerned about the positive and/or negative direction of 
washback, it is more plausible to consider the complexity and intensity 
of the phenomenon and explore the intricate causes of it in a given 
educational community. The success or failure of assessment-driven 
reform, they add, is not necessarily guaranteed beforehand. Rather, it 
will mostly depend on the inner dynamic of the system of education. 

In Chapter Two: 'Methodology in washback studies', Watanabe 
proposes a qualitative approach to investigating washback due to its 
complex rather than monolithic nature. He suggests a 
conceptualization of washback that centers round these dimensions: 
specificity, intensity, length, intentionality, and value, with a particular 
emphasis on analyzing the aspects of learning and teaching that are 
often influenced by the test, on one hand, and the factors that mediate 
the process of washback, including test facets, personal factors, and 
micro-/macro-contextual factors, on the other hand. He then provides 
detailed guidelines on how qualitative research on this area can be 
conducted, from designing research to selecting the participants, 
analyzing the data and interpreting the results. This chapter, it seems 
to me, is one of the best contributions to this collection. The points 
discussed in this chapter can lucidly serve as a set of criteria with 
reference to which one may judge the validity of the empirical studies 
on washback including those presented in the second part of the book.

Chapter Three: 'Washback and curriculum innovation' by Andrews, 
examines the assertions made about the nature of relationship that 
exists between curriculum innovation and washback. It offers a review 
of the available relevant empirical pieces of evidence for and against 
this link. Far from being a simple yes or no, he concludes, the 
influence of high-stakes tests on curriculum should be investigated 
through a careful examination of various niceties of the educational 
community and the changes introduced are often found to differ in 
type, depth, and complexity.

In Chapter Four: 'The effects of assessment-driven reform on the 
teaching of writing in Washington State', Stecher, Chun, and Barron 
report a study conducted to document the changes at both school and 
classroom levels during the early years of Washington Educational 
Reform which, among other things, introduced Washington 
Assessment of Student Learning (WASL) as a new assessment 
mechanism focusing on writing skill. Teachers and principals 
participating in the survey report changes in the allocation of time, 
emphasis on different aspects of learning, content and method of 
teaching, and students' learning activities. Replacing multiple-choice 
tests with more performance-based assessments appears to lead to 
an increase in the amount of writing students do in schools. Reliance 
on self-reports provided only by the teachers and principals, and not 
the students, nevertheless, can put this research and its findings in a 
rather vulnerable position.

Saville and Hawkey in Chapter Five, 'The IELTS impact study: 
Investigating washback on teaching materials', mostly present a 
research-in-progress report and focus on the data-collection 
instrument development phase of a large-scale, multi-phase IELTS 
impact study. They provide a detailed description of the instruments 
and procedures employed including user survey and structured group 
interviews with students, teachers and oral examiners, along with 
ratings of test practice books and materials. Early evidence points, it 
seems, to the authenticity of texts in test-related books as a beneficial 
effect of IELTS. In another study on the washback effect of IELTS 
reported in Chapter Six, 'IELTS test preparation in New Zealand', 
Hayes and Read attempt to examine whether IELTS test preparation 
courses in New Zealand show any evidence in support of washback 
related to this high-stakes test. Classroom observations, teacher 
interviews, teacher and student questionnaires, and pretests and 
posttests of students were the data collection tools used to investigate 
the nature of the two preparation courses. The obtained findings 
indicate drastic differences in the classroom practices offered to 
prepare students for IELTS implying that different instructors follow 
different methodologies to deal with the intended test tasks and 
materials. 

In a study of washback effect in the Australian Adult Migrant English 
Program reported in Chapter Seven, 'Washback in classroom-based 
assessments', Burrows documents the consequences of introducing 
the Certificate in Spoken and Written English (CSWE) backed up by 
data collected through questionnaires, interviews and classroom 
observations. The results of this study reveal that different teachers 
are affected differently by the new competency-based assessment 
system. In fact, Burrows categorizes the teachers' reaction to CSWE 
into four types: resister, adopter, partial adopter, and adaptor. In an 
interesting discussion, she criticizes the traditional view of washback 
which considers it to be a single, uniform response to a given test. 
Instead, she proposes a new model for washback "which takes into 
account teachers' belief systems and consequent responses to 
change" (p.125). A fruitful analysis, however, may address these 
observed variations in terms of their patterns of behavior rather than 
bewildering individual and idiosyncratic responses. It appears to me 
that the introduction of this model, which proposes seeking a pattern 
rather than being lost in the diversities, singles out this contribution as 
a turning point in the study of washback.

Results of the research into the washback effect of the English 
component of Japanese University Entrance Examination are 
presented in Chapter Eight, 'Teacher factors mediating washback'. In 
this study, Watanabe has collected the data through classroom 
observations and interviews with teachers. He concludes that the 
negative picture of the effect of the Entrance Examination as depicted 
by Japanese mass media does not truly reflect what is happening in 
the classrooms. The test, he adds, has both positive and negative 
washback effects. He also holds that the effect of the test is mediated 
by teachers' psychological factors and school cultures. Instead of a 
top-down approach to curriculum innovation, he calls for efforts 
directed towards some types of teacher training in order to introduce 
changes at the level of individual teachers. The author attracts our 
attention to many interesting points. However, as also pointed out by 
Watanabe himself "teachers were informed of the purpose of 
research" (p.133) and this can weaken the validity of his findings. 

In another study by Cheng reported in Chapter Nine, 'The washback 
effect of a public examination change on teachers' perceptions toward 
their classroom teaching', the washback effect of the new (1996) 
Hong Kong Certificate Examination in English (HKCEE) is investigated. 
HKCEE is designed to encourage more task-based teaching practices 
in Hong Kong. Analysis of teacher questionnaires and classroom 
observations reveal that teachers are reluctant to make fundamental 
changes in their daily practices, although their reactions to the test are 
positive. Based on these findings, Cheng concludes that changes in 
the educational system appear to be superficial rather than substantial 
and that a change in the examination alone is unlikely to fulfill the 
intended purposes of test designers and policy makers. These 
findings are further supported by Qi's study reported in Chapter 
Ten, 'Has a high-stakes test produced the intended changes?' It 
examines the intended washback effect of the National Matriculation 
English Test (NMET), a substitute for the old university entrance 
English examination in China. Through in-depth interviews and follow-
up contacts with the test constructors and teachers, Qi finds out that 
the test mainly influences the content of teaching but not teaching 
methodology employed by teachers. In fact, there appears to exist 
only a partial match between the test constructors' intention to 
promote communicative use of language and the reported actual 
classroom practices; hence, leading the author to the conclusion that 
a high-stakes test may not be a good lever for change. 

In Chapter Eleven, 'The washback effect of an EFL national 
matriculation test to teaching and learning', Ferman investigates the 
effect of a high-stakes test introduced as a means of curriculum 
innovation in the Israeli educational system. Drawing on the extensive 
data collected through multiple sources (structured questionnaires, 
structured interviews, open interviews, and document analysis) and 
multiple participants (teachers, EFL inspectors, as well as students), 
she finds out that there is a strong washback effect on the educational 
processes, products, and participants in Israeli high schools. The 
effects, however, are characterized as both positive (promotion of 
language skills, especially oral skills and different teaching/learning 
strategies) and negative (a higher level of anxiety and increased 
pressure to cover the materials). To me, Ferman's study is an 
impeccable research as it admirably meets the 'triangulation' criterion 
which guarantees the credibility of research (cf. Davies, 1995). 

Despite the fact that some of the studies reported in this collection 
suffer from certain methodological defects, I think, the collection is an 
important contribution to our understanding of the concept of 
washback in language testing. The reader gathers useful knowledge 
about washback and, at the same time, understands the ways 
contextual differences in different educational systems can affect the 
nature of washback in reality. The research reports edited in the 
volume, in my opinion, should be read critically as there are a number 
of shortcomings in the design and data collection procedure of a few 
studies that may limit the generalizability of their findings. 

REFERENCES

Alderson, Charles & Wall, Diane (1993) Does washback exist? Applied 
Linguistics 14, 115-129. 

Davies, Katharine (1995) Qualitative theory and methods in applied 
linguistics research. TESOL Quarterly 29, 427-453.

Messick, Samuel (1996) Validity and washback in language testing. 
Language Testing 13, 241-256. 

ABOUT THE REVIEWER

Esmat Babaii is an assistant professor in the Department of Foreign 
Languages at the University for Teacher Education, Tehran, Iran. She 
has taught graduate courses including language testing and research 
methods for several years. Her research interests include language 
testing, discourse analysis, EAP, and L2 research. She is currently the 
editor of the Asian EFL Journal and also a member of System review 
panel.





-----------------------------------------------------------
LINGUIST List: Vol-16-3304