31.1819, Review: Applied Linguistics: Rose, McKinley, Briggs Bafoe-Djan (2019)
The LINGUIST List
linguist at listserv.linguistlist.org
Tue Jun 2 15:58:23 UTC 2020
LINGUIST List: Vol-31-1819. Tue Jun 02 2020. ISSN: 1069 - 4875.
Subject: 31.1819, Review: Applied Linguistics: Rose, McKinley, Briggs Bafoe-Djan (2019)
Moderator: Malgorzata E. Cavar (linguist at linguistlist.org)
Student Moderator: Jeremy Coburn
Managing Editor: Becca Morris
Team: Helen Aristar-Dry, Everett Green, Sarah Robinson, Lauren Perkins, Nils Hjortnaes, Yiwen Zhang, Joshua Sims
Jobs: jobs at linguistlist.org | Conferences: callconf at linguistlist.org | Pubs: pubs at linguistlist.org
Homepage: http://linguistlist.org
Please support the LL editors and operation with a donation at:
https://funddrive.linguistlist.org/donate/
Editor for this issue: Jeremy Coburn <jecoburn at linguistlist.org>
================================================================
Date: Tue, 02 Jun 2020 11:57:57
From: Andrew Jocuns [jocunsa at gmail.com]
Subject: Data Collection Research Methods in Applied Linguistics
Discuss this message:
http://linguistlist.org/pubs/reviews/get-review.cfm?subid=36594577
Book announced at http://linguistlist.org/issues/30/30-4526.html
AUTHOR: Heath Rose
AUTHOR: Jim McKinley
AUTHOR: Jessica Briggs Bafoe-Djan
TITLE: Data Collection Research Methods in Applied Linguistics
SERIES TITLE: Research Methods in Linguistics
PUBLISHER: Bloomsbury Publishing (formerly The Continuum International Publishing Group)
YEAR: 2019
REVIEWER: Andrew Jocuns, Assumption University
SUMMARY
This book is the latest in Bloomsbury’s series Research Methods in Linguistics
and, as the authors note, this work is meant to be complementary to Paltridge
and Phakiti's (2015) Research Methods in Applied Linguistics. The difference
between them being that this work focuses upon data collection not research
design. Each chapter is structured similarly with sections on: pre-reading
tasks, key concepts, procedures and practicalities, improving validity and
reliability, examples of studies that use that method, implications for
researchers, post-reading activities, and resources for further reading.
Chapter 1, An Introduction to Research Methods covers some popular research
design methodologies including: experimental and quasi-experimental research,
field research, case study research, ethnographic research, survey research,
and corpus and document research. The authors provide a nice discussion of
quantitative and qualitative research including a discussion of mixed methods
designs that incorporate qualitative and quantitative data collection. Table
1.1 on page 15 juxtaposes the differences of qualitative and quantitative
research design, data collection and data analysis.
Data elicitation tasks are the focus of Chapter 2 and the authors note that
data elicitation has been widely employed within applied linguistics research.
Data elicitation tasks are divided into language production tasks (discourse
completion, role play, storytelling, oral proficiency interviews) and
experimental elicitation tasks (lexical decision, semantic categorization,
grammaticality judgement). Key concepts include: task naturalness, task
utility, task essentialness, modality, stimuli, fillers, counter-balancing and
priming. The authors identify a number of issues that researchers who
implement data elicitation tasks should keep in mind. For example, time
constraints can be a factor in collection because usually only one participant
can be interviewed at a time. In a subsection on stimuli for experimental
elicitation the authors offer a number of websites that host word generators.
The discussion on reliability and validation offers some suggestions for both
language production and experimental elicitation. The chapter concludes with a
discussion of seven applied linguistics studies which have used data
elicitation with a focused discussion on Grey et al. (2015) and Galante and
Thomson (2017), both of which implemented data elicitation in novel ways.
Introspective and retrospective protocols are taken up in Chapter 3, where
introspection involves participants looking inward on their own behaviors
(e.g. think aloud protocols) and retrospection involves a participant thinking
about their behavior during a past activity. While the authors argue that such
introspection is a valid protocol, they also suggest that one issue inherent
in this method is memory deterioration. To overcome this, they offer some
suggestions such as planning the data collection to occur as soon as possible
after the event. Think aloud protocols involve the participant voicing their
thoughts while performing a task and the authors note that a think aloud is
useful for writing research. Retrospective think aloud protocols are used
immediately after a participant is involved in a research task. Simulated
recall protocols are when stimuli are used to enhance the participants
recollection of events. Time constraints, researcher interference, reactivity
of the protocol, eye-tracking and keystroke logging are discussed in terms of
validity and reliability. Five sample studies are discussed at the end of the
chapter as well as implications which focus upon how in order to accurately
employ these types of methods researchers need to keep in mind factors that
can affect the protocol in terms of the task being studied.
Chapter 4 covers the use of validated tests and measures in applied
linguistics research where validated refers to the fact that there is an ample
body of research that the test or measure does what it claims to do. Key
concepts are broken down into three subcategories: types of tests and
measures, validity, and reliability. The authors also discuss procedures and
practicalities of choosing validated test and measures with an additional
dialogue on whether or not to adopt or adapt them into one’s study. The data
analysis is mentioned, in particular the focus on validity and reliability is
a very thorough and worthwhile read for graduate students or novice
researchers. Seven different studies are discussed with a nuanced focus upon
Briggs’ (2015) study of vocabulary acquisition in study abroad which used
Vocabulary Levels Test; and Saito et al. (2019) which used aptitude tests in a
study of language acquisition.
The focus of Chapter 5 is on observations as a form of data collection and a
number of important issues including the Hawthorne effect are discussed. Other
important discussions here include: the role(s) of the researcher,
self-observation, the importance of taking good field notes that involve thick
descriptions, as well as observation types and schemes. Reliability and
validity in observations are mentioned, which is on the surface dicey as most
observations are conducted by lone researchers. The authors suggest overcoming
this through having more than one observer. This is followed by discussions of
the Hawthorne effect when participants alter their behavior due to being
cognizant of being watched, as well as ways of improving validity and
reliability. Seven studies are showcased in the case studies section, all of
which used observation in different capacities, of note is
Curdt-Christiansen's (2016) study of family language policy which used
participant observations to examine and obtain detailed descriptions of family
language activity in practice.
Interviews are the topic of Chapter 6 where an emphasis is placed on the need
for researchers to take a reflexive stance in their conducting of interviews.
Reflexivity in interviews is derived from Talmy's (2010) discussions of the
interview as a form of social practice in which researchers refer to
themselves and their social influences during interviews. A discussion of
structured, semi-structured and unstructured interviews follows with
discussion of the effects of time (follow-up interviews) and interviewer bias.
The authors note that one of the biggest threats to interview reliability is a
lack of reflexivity on the part of the researcher, noting also that social
constructionism as a theory is another means of overcoming the lack of
reflexivity. Eight case studies that involved different types of interviews
were discussed but not in more nuanced detail. An interesting discussion
develops at the end of the chapter with regard to how interviews are better
perceived as discussion or conversations.
Chapter 7 draws our attention to the use of diaries, journals and logs as a
form of data where the participants are asked to keep track of certain topics
with regard to applied linguistics in their daily lives. This form of data,
the authors also note, has been largely underutilized but allows participants
to have agency and voice within the data. Different types of journals are
discussed as well as how they should be designed for the participants. The
discussion of reliability and validity emphasizes time and memory
deterioration. Another point of not is how participants often have difficulty
in writing in an L2, and to overcome this, researchers should allow
participants to write in their L1 and to allow for flexibility in terms of
entries in the journal through translanguaging and multimodality. Eight case
studies are noted emphasizing Galloway and Rose's (2014) use of 1092 listening
journal entries and Casanave's (2012) use of a journal that documented eight
years’ worth of studying and learning Japanese.
Questionnaires are the focus of Chapter 8. The authors point out early on how
novice researchers often align themselves with questionnaires under the
mistaken belief that this method is simple, noting of course the complexity
that is embedded with the design and implementation of questionnaires. The
chapter covers a lot of information that we typically see in methods books on
questionnaires (sampling, constructs, item writing, administration, piloting
and data analysis) with a focus on applied linguistics. Of those I found the
piloting discussion of particular importance for novice researchers. The
discussion on validity and reliability covers some important topics such as:
order effects, recall bias, and response rates. Six applied linguistics case
studies are highlighted with an emphasis on the previously noted Briggs (2015)
research on vocabulary learning in study and Teng and Zhang's (2016) study
which was focused upon validating a questionnaire.
Chapter 9, Focus Groups, discusses the various ways of designing, sampling,
constructing the participant makeup of, and moderating focus groups.
Emphasized here is how often in text books on methods focus groups are lumped
in as another interview type, when in reality they are a nuanced means of
gaining understanding of the depth and breadth of opinions and practices among
a group of people. The chapter includes an important discussion of choosing a
moderator who should not be the researcher, as well as dealing with
participants who are either too silent or the opposite; everyone’s voice needs
to be heard and dominant voices need to be handled early on so that can happen
effectively. Six applied linguistics studies that used focus groups are
discussed with a focus on Galloway's (2017) study, which used focus groups to
study the teaching of Global English to students in Japan, and Lanvers' (2018)
study, which used focus groups as the primary data such that the data were
representative of the target population. The authors provide a list of six
good practices for using focus groups on page 192.
Chapter 10 is on the use of documents as a form of data collection from
student writing to language policy documentation. One point of emphasis is on
how documents are socially constructed. The authors mention systematic reviews
which seek to evaluate the detailed findings of previous research and meta
analyses which reanalyzes data. These are juxtaposed with narrative reviews
which lack a clear research methodology. One interesting point the authors
make is that quantitative analyses of documents often require qualitative
analysis in order for the documents to have meaning. Document selection,
quality, and types of methodology (quantitative, qualitative and mixed method)
are also discussed. Eight case studies are discussed but none are focused
exclusively in this chapter.
Constructing and using corpora is the subject of Chapter 11 where the authors
effectively give us a crash course in corpus-based research. As someone who is
mildly familiar with corpus-based research, I found this chapter helpful in
thinking about and evaluating research that uses corpora. Types of corpora are
discussed as well as representativeness, balance and size as key concepts. As
far as procedures, the authors mention access as a key component of analyzing
previously constructed corpora. The chapter also includes discussion on:
transcription, processing text, software, marking up corpora, and how corpus
analysis is conducted using statistical means. Six studies are offered as
sample case studies that use corpora.
Chapter 12 is the final chapter and focuses upon strengthening data
collection. Triangulation is emphasized where using two or more sources to
validate data in order to have a comprehensive and accurate perspective on the
object of inquiry. One of the most interesting parts of the book was the
discussion in this chapter of replication and why studies are not replicated,
noting that replication of studies or instruments is really an open avenue for
applied linguistics research. It has never really occurred to me that there
was a lack of replication of studies and I am glad the authors brought this up
here. The chapter also offers a discussion of transparency in research noting
how it is often the case that how they were conducted and problems faced, are
often left out of journal articles or shelved in appendices. There are five
case studies presented in this chapter and they focus on transparency and
reflexivity in research as well as how technology was used in new ways, for
example audio recording using wristwatches (Seals & Kreeft Peyton, 2017).
EVALUATION
The audience for this text would be graduate students in applied linguistics
as well as novice and professional researchers. The book would be good for use
in an applied linguistics methods course. One point that the authors make that
is worthy of further discussion, and implementation, is the replication of
previous studies. One of the issues here is that many qualitative studies such
as open-ended interviews or ethnographic research, are research designs which
are not really replicable to the degree that experimental or clinical studies
are; however the methods used in data collection are. Overall this is a very
detailed book that covers data collection in applied linguistics. The only
weakness I note is where the authors discuss multimodality, which is it not
always as straightforward as it seems. I have found this book helpful in
thinking about how to advise graduate students on data collection in their
research and plan to use some of its chapters in a methods course.
REFERENCES
Briggs, J. G. (2015). Out-of-class language contact and vocabulary gain in a
study abroad context. System, 53, 129–140.
https://doi.org/10.1016/j.system.2015.07.007
Casanave, C. P. (2012). Diary of a Dabbler: Ecological Influences on an EFL
Teacher’s Efforts to Study Japanese Informally. TESOL Quarterly, 46(4),
642–670. https://doi.org/doi.org/10.1002/tesq.47
Curdt-Christiansen, X. L. (2016). Conflicting language ideologies and
contradictory language practices in Singaporean multilingual families. Journal
of Multilingual and Multicultural Development, 37(7), 694–709.
https://doi.org/10.1080/01434632.2015.1127926
Galante, A., & Thomson, R. I. (2017). The Effectiveness of Drama as an
Instructional Approach for the Development of Second Language Oral Fluency,
Comprehensibility, and Accentedness. TESOL Quarterly, 51(1), 115–142.
https://doi.org/10.1002/tesq.290
Galloway, N. (2017). Global Englishes and Change in English Language Teaching:
Attitudes and Impact. Routledge.
https://www.routledge.com/Global-Englishes-and-Change-in-English-Language-Teac
hing-Attitudes-and/Galloway/p/book/9780415786201
Galloway, N., & Rose, H. (2014). Using listening journals to raise awareness
of Global Englishes in ELT. ELT Journal, 68(4), 386–396.
https://doi.org/10.1093/elt/ccu021
Grey, S., Cox, J. G., Serafini, E. J., & Sanz, C. (2015). The Role of
Individual Differences in the Study Abroad Context: Cognitive Capacity and
Language Development During Short-Term Intensive Language Exposure. The Modern
Language Journal, 99(1), 137–157. https://doi.org/10.1111/modl.12190
Lanvers, U. (2018). ‘If they are going to university, they are gonna need a
language GCSE’: Co-constructing the social divide in language learning in
England. System, 76, 129–143. https://doi.org/10.1016/j.system.2018.05.010
Paltridge, B., & Phakiti, A. (2015). Research Methods in Applied Linguistics:
A Practical Resource.
Saito, K., Suzukida, Y., & Sun, H. (2019). Aptitude, experience, and second
language pronunciation proficiency development in classroom settings: A
longitudinal study. Studies in Second Language Acquisition, 41(1), 201–225.
https://doi.org/10.1017/S0272263117000432
Seals, C. A., & Kreeft Peyton, J. (2017). Heritage language education: Valuing
the languages, literacies, and cultural competencies of immigrant youth.
Current Issues in Language Planning, 18(1), 87–101.
https://doi.org/10.1080/14664208.2016.1168690
Talmy, S. (2010). Qualitative Interviews in Applied Linguistics: From Research
Instrument to Social Practice. Annual Review of Applied Linguistics, 30,
128–148. https://doi.org/10.1017/S0267190510000085
Talmy, S. (2011). The Interview as Collaborative Achievement: Interaction,
Identity, and Ideology in a Speech Event. Applied Linguistics, 32(1), 25–42.
https://doi.org/10.1093/applin/amq027
Teng, L. S., & Zhang, L. J. (2016). A Questionnaire-Based Validation of
Multidimensional Models of Self-Regulated Learning Strategies. The Modern
Language Journal, 100(3), 674–701. https://doi.org/10.1111/modl.12339
ABOUT THE REVIEWER
Andrew Jocuns, PhD, is a sociolinguist who presently holds<br />a position in
the English Language Teaching Program at<br />Assumption University of
Thailand. His most recent research has included a multimodal discourse
analysis of how students<br />perceive the linguistic landscape and
geosemiotics of their<br />schoolscape and their perceptions of Thai English.
------------------------------------------------------------------------------
*************************** LINGUIST List Support ***************************
The 2019 Fund Drive is under way! Please visit https://funddrive.linguistlist.org
to find out how to donate and check how your university, country or discipline
ranks in the fund drive challenges. Or go directly to the donation site:
https://iufoundation.fundly.com/the-linguist-list-2019
Let's make this a short fund drive!
Please feel free to share the link to our campaign:
https://funddrive.linguistlist.org/donate/
----------------------------------------------------------
LINGUIST List: Vol-31-1819
----------------------------------------------------------
More information about the LINGUIST
mailing list